You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@mesos.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/11/03 01:41:35 UTC
Build failed in Jenkins: Mesos » autotools,clang,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,ubuntu:14.04,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2873
See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%3A14.04,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-us1)&&(!ubuntu-6)/2873/changes>
Changes:
[josephwu] Fixed some markdown whitespace style.
[josephwu] Fixed a typo in `gpu-support.md`.
[josephwu] Fixed FD leak due to exiting early in Mesos fetcher.
------------------------------------------
[...truncated 78174 lines...]
I1103 01:40:55.530350 29098 slave.cpp:974] Authenticating with master master@172.17.0.2:58302
I1103 01:40:55.530432 29098 slave.cpp:985] Using default CRAM-MD5 authenticatee
I1103 01:40:55.530627 29098 slave.cpp:947] Detecting new master
I1103 01:40:55.530675 29108 authenticatee.cpp:121] Creating new client SASL connection
I1103 01:40:55.530743 29098 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
I1103 01:40:55.530961 29099 master.cpp:6742] Authenticating slave(150)@172.17.0.2:58302
I1103 01:40:55.531070 29112 authenticator.cpp:414] Starting authentication session for crammd5-authenticatee(357)@172.17.0.2:58302
I1103 01:40:55.531328 29106 authenticator.cpp:98] Creating new server SASL connection
I1103 01:40:55.531561 29108 authenticatee.cpp:213] Received SASL authentication mechanisms: CRAM-MD5
I1103 01:40:55.531604 29108 authenticatee.cpp:239] Attempting to authenticate with mechanism 'CRAM-MD5'
I1103 01:40:55.531713 29101 authenticator.cpp:204] Received SASL authentication start
I1103 01:40:55.531805 29101 authenticator.cpp:326] Authentication requires more steps
I1103 01:40:55.531921 29108 authenticatee.cpp:259] Received SASL authentication step
I1103 01:40:55.532120 29101 authenticator.cpp:232] Received SASL authentication step
I1103 01:40:55.532155 29101 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3a1c598ce334' server FQDN: '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
I1103 01:40:55.532179 29101 auxprop.cpp:181] Looking up auxiliary property '*userPassword'
I1103 01:40:55.532233 29101 auxprop.cpp:181] Looking up auxiliary property '*cmusaslsecretCRAM-MD5'
I1103 01:40:55.532266 29101 auxprop.cpp:109] Request to lookup properties for user: 'test-principal' realm: '3a1c598ce334' server FQDN: '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
I1103 01:40:55.532289 29101 auxprop.cpp:131] Skipping auxiliary property '*userPassword' since SASL_AUXPROP_AUTHZID == true
I1103 01:40:55.532305 29101 auxprop.cpp:131] Skipping auxiliary property '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
I1103 01:40:55.532335 29101 authenticator.cpp:318] Authentication success
I1103 01:40:55.532413 29110 authenticatee.cpp:299] Authentication success
I1103 01:40:55.532467 29108 master.cpp:6772] Successfully authenticated principal 'test-principal' at slave(150)@172.17.0.2:58302
I1103 01:40:55.532536 29111 authenticator.cpp:432] Authentication session cleanup for crammd5-authenticatee(357)@172.17.0.2:58302
I1103 01:40:55.532755 29098 slave.cpp:1069] Successfully authenticated with master master@172.17.0.2:58302
I1103 01:40:55.532997 29098 slave.cpp:1483] Will retry registration in 12.590371ms if necessary
I1103 01:40:55.533179 29108 master.cpp:5151] Registering agent at slave(150)@172.17.0.2:58302 (maintenance-host-2) with id 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
I1103 01:40:55.533572 29112 registrar.cpp:461] Applied 1 operations in 94467ns; attempting to update the registry
I1103 01:40:55.546341 29107 slave.cpp:1483] Will retry registration in 36.501523ms if necessary
I1103 01:40:55.546461 29099 master.cpp:5139] Ignoring register agent message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as admission is already in progress
I1103 01:40:55.565403 29097 leveldb.cpp:341] Persisting action (16 bytes) to leveldb took 48.099208ms
I1103 01:40:55.565495 29097 replica.cpp:708] Persisted action TRUNCATE at position 4
I1103 01:40:55.566788 29097 replica.cpp:691] Replica received learned notice for position 4 from @0.0.0.0:0
I1103 01:40:55.583937 29101 slave.cpp:1483] Will retry registration in 26.127711ms if necessary
I1103 01:40:55.584123 29112 master.cpp:5139] Ignoring register agent message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as admission is already in progress
I1103 01:40:55.609695 29097 leveldb.cpp:341] Persisting action (18 bytes) to leveldb took 42.905697ms
I1103 01:40:55.609860 29097 leveldb.cpp:399] Deleting ~2 keys from leveldb took 96623ns
I1103 01:40:55.609899 29097 replica.cpp:708] Persisted action TRUNCATE at position 4
I1103 01:40:55.611063 29106 log.cpp:577] Attempting to append 513 bytes to the log
I1103 01:40:55.611229 29097 coordinator.cpp:348] Coordinator attempting to write APPEND action at position 5
I1103 01:40:55.611498 29100 slave.cpp:1483] Will retry registration in 85.55417ms if necessary
I1103 01:40:55.612069 29105 master.cpp:5139] Ignoring register agent message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as admission is already in progress
I1103 01:40:55.612313 29100 replica.cpp:537] Replica received write request for position 5 from __req_res__(2518)@172.17.0.2:58302
I1103 01:40:55.657845 29100 leveldb.cpp:341] Persisting action (532 bytes) to leveldb took 45.517822ms
I1103 01:40:55.657938 29100 replica.cpp:708] Persisted action APPEND at position 5
I1103 01:40:55.658681 29103 replica.cpp:691] Replica received learned notice for position 5 from @0.0.0.0:0
I1103 01:40:55.698050 29112 slave.cpp:1483] Will retry registration in 100.32384ms if necessary
I1103 01:40:55.698319 29101 master.cpp:5139] Ignoring register agent message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as admission is already in progress
I1103 01:40:55.705086 29103 leveldb.cpp:341] Persisting action (534 bytes) to leveldb took 46.384557ms
I1103 01:40:55.705157 29103 replica.cpp:708] Persisted action APPEND at position 5
I1103 01:40:55.707480 29098 registrar.cpp:506] Successfully updated the registry in 173824us
I1103 01:40:55.707741 29100 log.cpp:596] Attempting to truncate the log to 5
I1103 01:40:55.708029 29110 coordinator.cpp:348] Coordinator attempting to write TRUNCATE action at position 6
I1103 01:40:55.708501 29097 slave.cpp:4251] Received ping from slave-observer(151)@172.17.0.2:58302
I1103 01:40:55.708528 29112 master.cpp:5222] Registered agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302 (maintenance-host-2) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
I1103 01:40:55.708796 29097 slave.cpp:1115] Registered with master master@172.17.0.2:58302; given agent ID 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
I1103 01:40:55.708822 29097 fetcher.cpp:86] Clearing fetcher cache
I1103 01:40:55.708889 29103 replica.cpp:537] Replica received write request for position 6 from __req_res__(2519)@172.17.0.2:58302
I1103 01:40:55.708935 29100 hierarchical.cpp:485] Added agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) with cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
I1103 01:40:55.709065 29105 status_update_manager.cpp:184] Resuming sending status updates
I1103 01:40:55.709139 29100 hierarchical.cpp:1694] No allocations performed
I1103 01:40:55.709204 29100 hierarchical.cpp:1309] Performed allocation for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 229932ns
I1103 01:40:55.709316 29097 slave.cpp:1138] Checkpointing SlaveInfo to '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/meta/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/slave.info'
I1103 01:40:55.709650 29097 slave.cpp:1175] Forwarding total oversubscribed resources {}
I1103 01:40:55.709787 29097 master.cpp:5621] Received update of agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302 (maintenance-host-2) with total oversubscribed resources {}
I1103 01:40:55.710000 29108 hierarchical.cpp:555] Agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) updated with oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000], allocated: {})
I1103 01:40:55.710144 29108 hierarchical.cpp:1694] No allocations performed
I1103 01:40:55.710199 29108 hierarchical.cpp:1309] Performed allocation for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 157491ns
I1103 01:40:55.711632 29105 process.cpp:3570] Handling HTTP event for process 'master' with path: '/master/maintenance/schedule'
I1103 01:40:55.712441 29111 http.cpp:391] HTTP POST for /master/maintenance/schedule from 172.17.0.2:45755
I1103 01:40:55.713325 29110 registrar.cpp:461] Applied 1 operations in 165622ns; attempting to update the registry
I1103 01:40:55.756808 29103 leveldb.cpp:341] Persisting action (16 bytes) to leveldb took 47.899715ms
I1103 01:40:55.756888 29103 replica.cpp:708] Persisted action TRUNCATE at position 6
I1103 01:40:55.758083 29103 replica.cpp:691] Replica received learned notice for position 6 from @0.0.0.0:0
I1103 01:40:55.807322 29103 leveldb.cpp:341] Persisting action (18 bytes) to leveldb took 49.23844ms
I1103 01:40:55.807481 29103 leveldb.cpp:399] Deleting ~2 keys from leveldb took 90240ns
I1103 01:40:55.807517 29103 replica.cpp:708] Persisted action TRUNCATE at position 6
I1103 01:40:55.808619 29101 log.cpp:577] Attempting to append 732 bytes to the log
I1103 01:40:55.808737 29097 coordinator.cpp:348] Coordinator attempting to write APPEND action at position 7
I1103 01:40:55.809571 29109 replica.cpp:537] Replica received write request for position 7 from __req_res__(2520)@172.17.0.2:58302
I1103 01:40:55.810784 29098 hierarchical.cpp:1694] No allocations performed
I1103 01:40:55.810860 29098 hierarchical.cpp:1286] Performed allocation for 2 agents in 282405ns
I1103 01:40:55.870082 29109 leveldb.cpp:341] Persisting action (751 bytes) to leveldb took 60.489282ms
I1103 01:40:55.870177 29109 replica.cpp:708] Persisted action APPEND at position 7
I1103 01:40:55.871421 29109 replica.cpp:691] Replica received learned notice for position 7 from @0.0.0.0:0
I1103 01:40:55.976619 29109 leveldb.cpp:341] Persisting action (753 bytes) to leveldb took 105.201942ms
I1103 01:40:55.976718 29109 replica.cpp:708] Persisted action APPEND at position 7
I1103 01:40:55.979708 29109 registrar.cpp:506] Successfully updated the registry in 266.301952ms
I1103 01:40:55.979878 29103 log.cpp:596] Attempting to truncate the log to 7
I1103 01:40:55.979991 29106 coordinator.cpp:348] Coordinator attempting to write TRUNCATE action at position 8
I1103 01:40:55.980255 29112 master.cpp:5683] Updating unavailability of agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302 (maintenance-host), starting at 2449.0107815972weeks
I1103 01:40:55.980481 29112 master.cpp:5683] Updating unavailability of agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302 (maintenance-host-2), starting at 2449.0107815972weeks
I1103 01:40:55.980818 29097 replica.cpp:537] Replica received write request for position 8 from __req_res__(2521)@172.17.0.2:58302
I1103 01:40:55.981092 29103 hierarchical.cpp:1694] No allocations performed
I1103 01:40:55.981150 29103 hierarchical.cpp:1309] Performed allocation for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 in 199516ns
I1103 01:40:55.981317 29103 hierarchical.cpp:1694] No allocations performed
I1103 01:40:55.981367 29103 hierarchical.cpp:1309] Performed allocation for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 144365ns
I1103 01:40:55.982864 29079 scheduler.cpp:176] Version: 1.2.0
I1103 01:40:55.983412 29112 scheduler.cpp:469] New master detected at master@172.17.0.2:58302
I1103 01:40:55.983438 29112 scheduler.cpp:478] Waiting for 0ns before initiating a re-(connection) attempt with the master
I1103 01:40:55.984688 29098 scheduler.cpp:353] Connected with the master at http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:55.985872 29109 scheduler.cpp:235] Sending SUBSCRIBE call to http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:55.986587 29099 process.cpp:3570] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1103 01:40:55.987659 29108 http.cpp:391] HTTP POST for /master/api/v1/scheduler from 172.17.0.2:45771
I1103 01:40:55.988019 29108 master.cpp:2329] Received subscription request for HTTP framework 'default'
I1103 01:40:55.988088 29108 master.cpp:2069] Authorizing framework principal 'test-principal' to receive offers for role '*'
I1103 01:40:55.988632 29099 master.cpp:2427] Subscribing framework 'default' with checkpointing disabled and capabilities [ ]
I1103 01:40:55.989305 29105 hierarchical.cpp:275] Added framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:55.989459 29099 master.hpp:2161] Sending heartbeat to 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:55.990151 29108 scheduler.cpp:675] Enqueuing event SUBSCRIBED received from http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:55.990648 29108 scheduler.cpp:675] Enqueuing event HEARTBEAT received from http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:55.991364 29105 hierarchical.cpp:1286] Performed allocation for 2 agents in 2.044414ms
I1103 01:40:55.992657 29100 master.cpp:6571] Sending 2 offers to framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
I1103 01:40:55.994067 29100 master.cpp:6661] Sending 2 inverse offers to framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
I1103 01:40:55.995620 29101 scheduler.cpp:675] Enqueuing event OFFERS received from http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:55.996536 29101 scheduler.cpp:675] Enqueuing event INVERSE_OFFERS received from http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:56.060276 29097 leveldb.cpp:341] Persisting action (16 bytes) to leveldb took 79.452553ms
I1103 01:40:56.060681 29097 replica.cpp:708] Persisted action TRUNCATE at position 8
I1103 01:40:56.061957 29102 replica.cpp:691] Replica received learned notice for position 8 from @0.0.0.0:0
I1103 01:40:56.118927 29102 leveldb.cpp:341] Persisting action (18 bytes) to leveldb took 56.925572ms
I1103 01:40:56.119022 29102 leveldb.cpp:399] Deleting ~2 keys from leveldb took 51598ns
I1103 01:40:56.119046 29102 replica.cpp:708] Persisted action TRUNCATE at position 8
I1103 01:40:56.120704 29099 hierarchical.cpp:1694] No allocations performed
I1103 01:40:56.121099 29099 hierarchical.cpp:1286] Performed allocation for 2 agents in 610047ns
I1103 01:40:56.122525 29097 scheduler.cpp:235] Sending ACCEPT call to http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:56.123138 29097 scheduler.cpp:235] Sending ACCEPT call to http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:56.124460 29098 process.cpp:3570] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1103 01:40:56.124822 29098 process.cpp:3570] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1103 01:40:56.126590 29098 http.cpp:391] HTTP POST for /master/api/v1/scheduler from 172.17.0.2:45772
I1103 01:40:56.127682 29098 master.cpp:3581] Processing ACCEPT call for offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O0 ] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302 (maintenance-host) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
I1103 01:40:56.127856 29098 master.cpp:3173] Authorizing framework principal 'test-principal' to launch task 19b41cf9-ba0f-472e-8cae-5ea70574b1b4
I1103 01:40:56.128916 29098 http.cpp:391] HTTP POST for /master/api/v1/scheduler from 172.17.0.2:45772
I1103 01:40:56.129622 29098 master.cpp:3581] Processing ACCEPT call for offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O1 ] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302 (maintenance-host-2) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
I1103 01:40:56.129730 29098 master.cpp:3173] Authorizing framework principal 'test-principal' to launch task 846a812b-b562-4782-b0fd-9e760a52306e
W1103 01:40:56.131443 29098 validation.cpp:920] Executor 'executor-1' for task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less CPUs (None) than the minimum required (0.01). Please update your executor, as this will be mandatory in future releases.
W1103 01:40:56.131489 29098 validation.cpp:932] Executor 'executor-1' for task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less memory (None) than the minimum required (32MB). Please update your executor, as this will be mandatory in future releases.
I1103 01:40:56.132071 29098 master.cpp:8334] Adding task 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 with resources cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 (maintenance-host)
I1103 01:40:56.132442 29098 master.cpp:4230] Launching task 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) with resources cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302 (maintenance-host)
W1103 01:40:56.133824 29098 validation.cpp:920] Executor 'executor-2' for task '846a812b-b562-4782-b0fd-9e760a52306e' uses less CPUs (None) than the minimum required (0.01). Please update your executor, as this will be mandatory in future releases.
W1103 01:40:56.133858 29098 validation.cpp:932] Executor 'executor-2' for task '846a812b-b562-4782-b0fd-9e760a52306e' uses less memory (None) than the minimum required (32MB). Please update your executor, as this will be mandatory in future releases.
I1103 01:40:56.134299 29097 slave.cpp:1547] Got assigned task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.134332 29098 master.cpp:8334] Adding task 846a812b-b562-4782-b0fd-9e760a52306e with resources cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2)
I1103 01:40:56.134572 29098 master.cpp:4230] Launching task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) with resources cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302 (maintenance-host-2)
I1103 01:40:56.135084 29097 slave.cpp:1709] Launching task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.135145 29098 slave.cpp:1547] Got assigned task '846a812b-b562-4782-b0fd-9e760a52306e' for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.135660 29098 slave.cpp:1709] Launching task '846a812b-b562-4782-b0fd-9e760a52306e' for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.135871 29097 paths.cpp:536] Trying to chown '/tmp/MasterMaintenanceTest_InverseOffersFilters_B4KX1H/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S0/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-1/runs/fbb81e53-b134-4196-9385-791612794545' to user 'mesos'
I1103 01:40:56.138550 29098 paths.cpp:536] Trying to chown '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4' to user 'mesos'
I1103 01:40:56.144282 29098 slave.cpp:6307] Launching executor 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 with resources {} in work directory '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
I1103 01:40:56.144282 29097 slave.cpp:6307] Launching executor 'executor-1' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 with resources {} in work directory '/tmp/MasterMaintenanceTest_InverseOffersFilters_B4KX1H/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S0/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-1/runs/fbb81e53-b134-4196-9385-791612794545'
F1103 01:40:56.144891 29097 owned.hpp:110] Check failed: 'get()' Must be non NULL
*** Check failure stack trace: ***
I1103 01:40:56.147233 29098 exec.cpp:162] Version: 1.2.0
I1103 01:40:56.147598 29112 exec.cpp:212] Executor started at: executor(60)@172.17.0.2:58302 with pid 29079
I1103 01:40:56.147956 29098 slave.cpp:2031] Queued task '846a812b-b562-4782-b0fd-9e760a52306e' for executor 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.148043 29098 slave.cpp:868] Successfully attached file '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
I1103 01:40:56.148277 29098 slave.cpp:3305] Got registration for executor 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from executor(60)@172.17.0.2:58302
I1103 01:40:56.148833 29105 exec.cpp:237] Executor registered on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
I1103 01:40:56.148895 29105 exec.cpp:249] Executor::registered took 29894ns
I1103 01:40:56.149114 29098 slave.cpp:2247] Sending queued task '846a812b-b562-4782-b0fd-9e760a52306e' to executor 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 at executor(60)@172.17.0.2:58302
I1103 01:40:56.149495 29106 exec.cpp:324] Executor asked to run task '846a812b-b562-4782-b0fd-9e760a52306e'
I1103 01:40:56.149608 29106 exec.cpp:333] Executor::launchTask took 98162ns
I1103 01:40:56.149705 29106 exec.cpp:550] Executor sending status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.149987 29106 slave.cpp:3740] Handling status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from executor(60)@172.17.0.2:58302
I1103 01:40:56.150545 29098 status_update_manager.cpp:323] Received status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.150606 29098 status_update_manager.cpp:500] Creating StatusUpdate stream for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.151116 29098 status_update_manager.cpp:377] Forwarding update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to the agent
@ 0x2b56852902fd google::LogMessage::Fail()
I1103 01:40:56.151470 29101 slave.cpp:4169] Forwarding the update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to master@172.17.0.2:58302
I1103 01:40:56.151757 29101 slave.cpp:4063] Status update manager successfully handled status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.151860 29101 slave.cpp:4079] Sending acknowledgement for status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to executor(60)@172.17.0.2:58302
I1103 01:40:56.151954 29111 master.cpp:5757] Status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302 (maintenance-host-2)
I1103 01:40:56.152041 29111 master.cpp:5819] Forwarding status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.152123 29100 exec.cpp:373] Executor received status update acknowledgement 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.152546 29111 master.cpp:7712] Updating the state of task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
I1103 01:40:56.153616 29103 scheduler.cpp:675] Enqueuing event UPDATE received from http://172.17.0.2:58302/master/api/v1/scheduler
@ 0x2b568528f6de google::LogMessage::SendToLog()
I1103 01:40:56.164335 29109 scheduler.cpp:235] Sending ACKNOWLEDGE call to http://172.17.0.2:58302/master/api/v1/scheduler
I1103 01:40:56.165396 29109 process.cpp:3570] Handling HTTP event for process 'master' with path: '/master/api/v1/scheduler'
I1103 01:40:56.167019 29109 http.cpp:391] HTTP POST for /master/api/v1/scheduler from 172.17.0.2:45772
I1103 01:40:56.167212 29109 master.cpp:4867] Processing ACKNOWLEDGE call 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
I1103 01:40:56.168145 29110 status_update_manager.cpp:395] Received status update acknowledgement (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
I1103 01:40:56.168666 29110 slave.cpp:3022] Status update manager successfully handled status update acknowledgement (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
@ 0x2b568528ffbd google::LogMessage::Flush()
@ 0x2b5685293438 google::LogMessageFatal::~LogMessageFatal()
@ 0xcb1fc4 google::CheckNotNull<>()
@ 0xc93ba6 process::Owned<>::operator->()
@ 0xc9100c mesos::internal::tests::TestContainerizer::_launch()
@ 0xc98182 _ZN7testing8internal12InvokeHelperIN7process6FutureIbEESt5tupleIJRKN5mesos11ContainerIDERK6OptionINS6_8TaskInfoEERKNS6_12ExecutorInfoERKSsRKSA_ISsERKNS6_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISI_SsEEEbEEE12InvokeMethodINS6_8internal5tests17TestContainerizerEMS14_FS4_S9_SE_SH_SJ_SM_SP_SY_bEEES4_PT_T0_RKSZ_
@ 0xc98014 _ZNK7testing8internal18InvokeMethodActionIN5mesos8internal5tests17TestContainerizerEMS5_FN7process6FutureIbEERKNS2_11ContainerIDERK6OptionINS2_8TaskInfoEERKNS2_12ExecutorInfoERKSsRKSC_ISsERKNS2_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISK_SsEEEbEE7PerformIS8_St5tupleIJSB_SG_SJ_SL_SO_SR_S10_bEEEET_RKT0_
@ 0xc97fb6 _ZN7testing17PolymorphicActionINS_8internal18InvokeMethodActionIN5mesos8internal5tests17TestContainerizerEMS6_FN7process6FutureIbEERKNS3_11ContainerIDERK6OptionINS3_8TaskInfoEERKNS3_12ExecutorInfoERKSsRKSD_ISsERKNS3_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISL_SsEEEbEEEE15MonomorphicImplIFS9_SC_SH_SK_SM_SP_SS_S11_bEE7PerformERKSt5tupleIJSC_SH_SK_SM_SP_SS_S11_bEE
@ 0xb51d58 _ZNK7testing6ActionIFN7process6FutureIbEERKN5mesos11ContainerIDERK6OptionINS4_8TaskInfoEERKNS4_12ExecutorInfoERKSsRKS8_ISsERKNS4_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISG_SsEEEbEE7PerformERKSt5tupleIJS7_SC_SF_SH_SK_SN_SW_bEE
@ 0xb51b6a testing::internal::ActionResultHolder<>::PerformAction<>()
@ 0xb4ca1c testing::internal::FunctionMockerBase<>::UntypedPerformAction()
@ 0x1daa619 testing::internal::UntypedFunctionMockerBase::UntypedInvokeWith()
@ 0xb4995b _ZN7testing8internal18FunctionMockerBaseIFN7process6FutureIbEERKN5mesos11ContainerIDERK6OptionINS5_8TaskInfoEERKNS5_12ExecutorInfoERKSsRKS9_ISsERKNS5_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISH_SsEEEbEE10InvokeWithERKSt5tupleIJS8_SD_SG_SI_SL_SO_SX_bEE
@ 0xb49916 testing::internal::FunctionMocker<>::Invoke()
@ 0xc9712f mesos::internal::tests::TestContainerizer::launch()
@ 0x2b5684298b9b mesos::internal::slave::Framework::launchExecutor()
@ 0x2b568429607e mesos::internal::slave::Slave::_run()
@ 0x2b56843265a0 _ZZN7process8dispatchIN5mesos8internal5slave5SlaveERKNS_6FutureIbEERKNS1_13FrameworkInfoERKNS1_12ExecutorInfoERK6OptionINS1_8TaskInfoEERKSF_INS1_13TaskGroupInfoEES6_S9_SC_SH_SL_EEvRKNS_3PIDIT_EEMSP_FvT0_T1_T2_T3_T4_ET5_T6_T7_T8_T9_ENKUlPNS_11ProcessBaseEE_clES16_
@ 0x2b5684326062 _ZNSt17_Function_handlerIFvPN7process11ProcessBaseEEZNS0_8dispatchIN5mesos8internal5slave5SlaveERKNS0_6FutureIbEERKNS5_13FrameworkInfoERKNS5_12ExecutorInfoERK6OptionINS5_8TaskInfoEERKSJ_INS5_13TaskGroupInfoEESA_SD_SG_SL_SP_EEvRKNS0_3PIDIT_EEMST_FvT0_T1_T2_T3_T4_ET5_T6_T7_T8_T9_EUlS2_E_E9_M_invokeERKSt9_Any_dataS2_
@ 0x2b56851b2dd8 std::function<>::operator()()
@ 0x2b5685199f74 process::ProcessBase::visit()
@ 0x2b56851f84ee process::DispatchEvent::visit()
@ 0x8dabe1 process::ProcessBase::serve()
@ 0x2b5685197c64 process::ProcessManager::resume()
@ 0x2b56851a28cc process::ProcessManager::init_threads()::$_0::operator()()
@ 0x2b56851a27d5 _ZNSt12_Bind_simpleIFZN7process14ProcessManager12init_threadsEvE3$_0vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE
@ 0x2b56851a27a5 std::_Bind_simple<>::operator()()
@ 0x2b56851a277c std::thread::_Impl<>::_M_run()
@ 0x2b5686d10a60 (unknown)
@ 0x2b5687487184 start_thread
make[4]: *** [check-local] Aborted
make[4]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
make[3]: *** [check-am] Error 2
make[3]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
make[2]: *** [check] Error 2
make[2]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/mesos/mesos-1.2.0/_build'
make: *** [distcheck] Error 1
+ docker rmi mesos-1478134041-23345
Untagged: mesos-1478134041-23345:latest
Deleted: sha256:74596d8e8b76315d1115980623d7e5af9150c10d54203c726895aa5cac6da79e
Deleted: sha256:7a0b0a4cf4ac2c9897b51052cb68965f0b4e8a305ff6db16b7d530709b6e406c
Deleted: sha256:567f543714141c74573998d4209337287ffbe0e8ccd9ee8e4b044cc6e872c0e5
Deleted: sha256:fbd8be03b0d3f1c70dea858388d951374325b608d3e56c6b51fd102b193afed2
Deleted: sha256:7c7aee38843610c581d675c5620d8981f65d7fda18fc0fa05a264fb510dd927c
Deleted: sha256:654d69e9689193c66be28a0fff218d21a5a119fbeecdc26a8a4cf34b3c2cd104
Deleted: sha256:4f5e0007d3903ea948c9fa13032fad2b1244dc0d4dbbee0673cd3470c8f3c24a
Deleted: sha256:38239a48898a69f2bf752d732a101a22038fc2d8e023b6c789987daa2332ee1b
Deleted: sha256:329c7b62b50ac8578b3499838a1f12435013f0b4f429da2efac3d76673800950
Deleted: sha256:45dc8a92ffad997c6e48598af8e4665ff06cbb6d6316351de710317f420367a2
Deleted: sha256:bc4e9bad738f327a22d9bb1204cbb3787b6b04fcda850497d7a6fd54b1d9b9f3
Deleted: sha256:708772f854104da0e8b60eb0a9fb12c5cd85fca258ed74289e0cadcab12bb2f9
Deleted: sha256:5498b1ca089ca60b79713f6a57bc63036a89079ae1afff2ec21d442d5fa531f1
Deleted: sha256:7830805a402ac02a943fc9dbac0ff1dd46c4cfd67af2fb4e36b9d2e43dd7a4d5
Deleted: sha256:354a09c0839db0533726726c863f23138aad1656c38ee8f3c16b04b1766a0109
Deleted: sha256:73999f055e575a9767df52d22b67980d2b536b8ac1eeda0e9af1ea623fc939b9
Deleted: sha256:c915f730bb38299ca517a3f295fcdd21e0078d959381ae5a9aff0f39b4748bd5
Deleted: sha256:ca688ce3b2900d35c52872f38fd4af8530b780bb362e6e1a42687d4f5e4082e8
Deleted: sha256:d2c33d1bb8213d5ed68c888aa09a65370a36d6ab4311704d2f106e7ef94cb6fc
Deleted: sha256:67e4c09745aa1fd979f24bfae0812811bd44377dafd43130c0c2769e820d252a
Deleted: sha256:958624053829cf5b33921f84ad226d1835a271befa6856f591fc0b4e1981c349
Deleted: sha256:a59579b1b2fd24a0570169c6a97d2ba86c88f44a9ac159c792f3771c9d3585a7
Deleted: sha256:4edbd39dc9e6dcb72d14b8e207b6705bace589ce1dc8cfe4525ffac3d216f471
Deleted: sha256:92fc49f6ea70503f089441fdcdfdcf8ebc3875060deefaf5f355c4a9b45314e8
Build step 'Execute shell' marked build as failure
Jenkins build is back to normal : Mesos » autotools,clang,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,ubuntu:14.04,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2874
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%3A14.04,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-us1)&&(!ubuntu-6)/2874/changes>
Re: Build failed in Jenkins: Mesos » autotools,clang,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,ubuntu:14.04,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2873
Posted by Benjamin Mahler <bm...@apache.org>.
Fix is here: https://reviews.apache.org/r/53455/
On Thu, Nov 3, 2016 at 5:11 PM, Benjamin Mahler <bm...@apache.org> wrote:
> https://issues.apache.org/jira/browse/MESOS-6544
> https://issues.apache.org/jira/browse/MESOS-6545
>
> On Thu, Nov 3, 2016 at 4:56 PM, Benjamin Mahler <bm...@apache.org>
> wrote:
>
>> This test uses two mock executors across two agents, which means the
>> agents may concurrently call into the TestContainerizer to launch the
>> container. The TestContainerizer is not a Process and so we may
>> concurrently modify its data structures, I suspect that the following
>> executes concurrently and leads to the crash:
>>
>> https://github.com/apache/mesos/blob/44242c058158727ce013bd5
>> 1764368f5e120ee75/src/tests/containerizer.cpp#L131
>>
>> On Wed, Nov 2, 2016 at 6:41 PM, Apache Jenkins Server <
>> jenkins@builds.apache.org> wrote:
>>
>>> See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,COM
>>> PILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20--
>>> enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubuntu%
>>> 3A14.04,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-us1)&&(!
>>> ubuntu-6)/2873/changes>
>>>
>>> Changes:
>>>
>>> [josephwu] Fixed some markdown whitespace style.
>>>
>>> [josephwu] Fixed a typo in `gpu-support.md`.
>>>
>>> [josephwu] Fixed FD leak due to exiting early in Mesos fetcher.
>>>
>>> ------------------------------------------
>>> [...truncated 78174 lines...]
>>> I1103 01:40:55.530350 29098 slave.cpp:974] Authenticating with master
>>> master@172.17.0.2:58302
>>> I1103 01:40:55.530432 29098 slave.cpp:985] Using default CRAM-MD5
>>> authenticatee
>>> I1103 01:40:55.530627 29098 slave.cpp:947] Detecting new master
>>> I1103 01:40:55.530675 29108 authenticatee.cpp:121] Creating new client
>>> SASL connection
>>> I1103 01:40:55.530743 29098 slave.cpp:5587] Received oversubscribable
>>> resources {} from the resource estimator
>>> I1103 01:40:55.530961 29099 master.cpp:6742] Authenticating slave(150)@
>>> 172.17.0.2:58302
>>> I1103 01:40:55.531070 29112 authenticator.cpp:414] Starting
>>> authentication session for crammd5-authenticatee(357)@172.17.0.2:58302
>>> I1103 01:40:55.531328 29106 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I1103 01:40:55.531561 29108 authenticatee.cpp:213] Received SASL
>>> authentication mechanisms: CRAM-MD5
>>> I1103 01:40:55.531604 29108 authenticatee.cpp:239] Attempting to
>>> authenticate with mechanism 'CRAM-MD5'
>>> I1103 01:40:55.531713 29101 authenticator.cpp:204] Received SASL
>>> authentication start
>>> I1103 01:40:55.531805 29101 authenticator.cpp:326] Authentication
>>> requires more steps
>>> I1103 01:40:55.531921 29108 authenticatee.cpp:259] Received SASL
>>> authentication step
>>> I1103 01:40:55.532120 29101 authenticator.cpp:232] Received SASL
>>> authentication step
>>> I1103 01:40:55.532155 29101 auxprop.cpp:109] Request to lookup
>>> properties for user: 'test-principal' realm: '3a1c598ce334' server FQDN:
>>> '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
>>> I1103 01:40:55.532179 29101 auxprop.cpp:181] Looking up auxiliary
>>> property '*userPassword'
>>> I1103 01:40:55.532233 29101 auxprop.cpp:181] Looking up auxiliary
>>> property '*cmusaslsecretCRAM-MD5'
>>> I1103 01:40:55.532266 29101 auxprop.cpp:109] Request to lookup
>>> properties for user: 'test-principal' realm: '3a1c598ce334' server FQDN:
>>> '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
>>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
>>> I1103 01:40:55.532289 29101 auxprop.cpp:131] Skipping auxiliary property
>>> '*userPassword' since SASL_AUXPROP_AUTHZID == true
>>> I1103 01:40:55.532305 29101 auxprop.cpp:131] Skipping auxiliary property
>>> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
>>> I1103 01:40:55.532335 29101 authenticator.cpp:318] Authentication success
>>> I1103 01:40:55.532413 29110 authenticatee.cpp:299] Authentication success
>>> I1103 01:40:55.532467 29108 master.cpp:6772] Successfully authenticated
>>> principal 'test-principal' at slave(150)@172.17.0.2:58302
>>> I1103 01:40:55.532536 29111 authenticator.cpp:432] Authentication
>>> session cleanup for crammd5-authenticatee(357)@172.17.0.2:58302
>>> I1103 01:40:55.532755 29098 slave.cpp:1069] Successfully authenticated
>>> with master master@172.17.0.2:58302
>>> I1103 01:40:55.532997 29098 slave.cpp:1483] Will retry registration in
>>> 12.590371ms if necessary
>>> I1103 01:40:55.533179 29108 master.cpp:5151] Registering agent at
>>> slave(150)@172.17.0.2:58302 (maintenance-host-2) with id
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
>>> I1103 01:40:55.533572 29112 registrar.cpp:461] Applied 1 operations in
>>> 94467ns; attempting to update the registry
>>> I1103 01:40:55.546341 29107 slave.cpp:1483] Will retry registration in
>>> 36.501523ms if necessary
>>> I1103 01:40:55.546461 29099 master.cpp:5139] Ignoring register agent
>>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>>> admission is already in progress
>>> I1103 01:40:55.565403 29097 leveldb.cpp:341] Persisting action (16
>>> bytes) to leveldb took 48.099208ms
>>> I1103 01:40:55.565495 29097 replica.cpp:708] Persisted action TRUNCATE
>>> at position 4
>>> I1103 01:40:55.566788 29097 replica.cpp:691] Replica received learned
>>> notice for position 4 from @0.0.0.0:0
>>> I1103 01:40:55.583937 29101 slave.cpp:1483] Will retry registration in
>>> 26.127711ms if necessary
>>> I1103 01:40:55.584123 29112 master.cpp:5139] Ignoring register agent
>>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>>> admission is already in progress
>>> I1103 01:40:55.609695 29097 leveldb.cpp:341] Persisting action (18
>>> bytes) to leveldb took 42.905697ms
>>> I1103 01:40:55.609860 29097 leveldb.cpp:399] Deleting ~2 keys from
>>> leveldb took 96623ns
>>> I1103 01:40:55.609899 29097 replica.cpp:708] Persisted action TRUNCATE
>>> at position 4
>>> I1103 01:40:55.611063 29106 log.cpp:577] Attempting to append 513 bytes
>>> to the log
>>> I1103 01:40:55.611229 29097 coordinator.cpp:348] Coordinator attempting
>>> to write APPEND action at position 5
>>> I1103 01:40:55.611498 29100 slave.cpp:1483] Will retry registration in
>>> 85.55417ms if necessary
>>> I1103 01:40:55.612069 29105 master.cpp:5139] Ignoring register agent
>>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>>> admission is already in progress
>>> I1103 01:40:55.612313 29100 replica.cpp:537] Replica received write
>>> request for position 5 from __req_res__(2518)@172.17.0.2:58302
>>> I1103 01:40:55.657845 29100 leveldb.cpp:341] Persisting action (532
>>> bytes) to leveldb took 45.517822ms
>>> I1103 01:40:55.657938 29100 replica.cpp:708] Persisted action APPEND at
>>> position 5
>>> I1103 01:40:55.658681 29103 replica.cpp:691] Replica received learned
>>> notice for position 5 from @0.0.0.0:0
>>> I1103 01:40:55.698050 29112 slave.cpp:1483] Will retry registration in
>>> 100.32384ms if necessary
>>> I1103 01:40:55.698319 29101 master.cpp:5139] Ignoring register agent
>>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>>> admission is already in progress
>>> I1103 01:40:55.705086 29103 leveldb.cpp:341] Persisting action (534
>>> bytes) to leveldb took 46.384557ms
>>> I1103 01:40:55.705157 29103 replica.cpp:708] Persisted action APPEND at
>>> position 5
>>> I1103 01:40:55.707480 29098 registrar.cpp:506] Successfully updated the
>>> registry in 173824us
>>> I1103 01:40:55.707741 29100 log.cpp:596] Attempting to truncate the log
>>> to 5
>>> I1103 01:40:55.708029 29110 coordinator.cpp:348] Coordinator attempting
>>> to write TRUNCATE action at position 6
>>> I1103 01:40:55.708501 29097 slave.cpp:4251] Received ping from
>>> slave-observer(151)@172.17.0.2:58302
>>> I1103 01:40:55.708528 29112 master.cpp:5222] Registered agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>>> (maintenance-host-2) with cpus(*):2; mem(*):1024; disk(*):1024;
>>> ports(*):[31000-32000]
>>> I1103 01:40:55.708796 29097 slave.cpp:1115] Registered with master
>>> master@172.17.0.2:58302; given agent ID 3167a687-904b-4b57-bc0f-91b67d
>>> c7e41d-S1
>>> I1103 01:40:55.708822 29097 fetcher.cpp:86] Clearing fetcher cache
>>> I1103 01:40:55.708889 29103 replica.cpp:537] Replica received write
>>> request for position 6 from __req_res__(2519)@172.17.0.2:58302
>>> I1103 01:40:55.708935 29100 hierarchical.cpp:485] Added agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) with
>>> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
>>> I1103 01:40:55.709065 29105 status_update_manager.cpp:184] Resuming
>>> sending status updates
>>> I1103 01:40:55.709139 29100 hierarchical.cpp:1694] No allocations
>>> performed
>>> I1103 01:40:55.709204 29100 hierarchical.cpp:1309] Performed allocation
>>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 229932ns
>>> I1103 01:40:55.709316 29097 slave.cpp:1138] Checkpointing SlaveInfo to
>>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/meta
>>> /slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/slave.info'
>>> I1103 01:40:55.709650 29097 slave.cpp:1175] Forwarding total
>>> oversubscribed resources {}
>>> I1103 01:40:55.709787 29097 master.cpp:5621] Received update of agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>>> (maintenance-host-2) with total oversubscribed resources {}
>>> I1103 01:40:55.710000 29108 hierarchical.cpp:555] Agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) updated
>>> with oversubscribed resources {} (total: cpus(*):2; mem(*):1024;
>>> disk(*):1024; ports(*):[31000-32000], allocated: {})
>>> I1103 01:40:55.710144 29108 hierarchical.cpp:1694] No allocations
>>> performed
>>> I1103 01:40:55.710199 29108 hierarchical.cpp:1309] Performed allocation
>>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 157491ns
>>> I1103 01:40:55.711632 29105 process.cpp:3570] Handling HTTP event for
>>> process 'master' with path: '/master/maintenance/schedule'
>>> I1103 01:40:55.712441 29111 http.cpp:391] HTTP POST for
>>> /master/maintenance/schedule from 172.17.0.2:45755
>>> I1103 01:40:55.713325 29110 registrar.cpp:461] Applied 1 operations in
>>> 165622ns; attempting to update the registry
>>> I1103 01:40:55.756808 29103 leveldb.cpp:341] Persisting action (16
>>> bytes) to leveldb took 47.899715ms
>>> I1103 01:40:55.756888 29103 replica.cpp:708] Persisted action TRUNCATE
>>> at position 6
>>> I1103 01:40:55.758083 29103 replica.cpp:691] Replica received learned
>>> notice for position 6 from @0.0.0.0:0
>>> I1103 01:40:55.807322 29103 leveldb.cpp:341] Persisting action (18
>>> bytes) to leveldb took 49.23844ms
>>> I1103 01:40:55.807481 29103 leveldb.cpp:399] Deleting ~2 keys from
>>> leveldb took 90240ns
>>> I1103 01:40:55.807517 29103 replica.cpp:708] Persisted action TRUNCATE
>>> at position 6
>>> I1103 01:40:55.808619 29101 log.cpp:577] Attempting to append 732 bytes
>>> to the log
>>> I1103 01:40:55.808737 29097 coordinator.cpp:348] Coordinator attempting
>>> to write APPEND action at position 7
>>> I1103 01:40:55.809571 29109 replica.cpp:537] Replica received write
>>> request for position 7 from __req_res__(2520)@172.17.0.2:58302
>>> I1103 01:40:55.810784 29098 hierarchical.cpp:1694] No allocations
>>> performed
>>> I1103 01:40:55.810860 29098 hierarchical.cpp:1286] Performed allocation
>>> for 2 agents in 282405ns
>>> I1103 01:40:55.870082 29109 leveldb.cpp:341] Persisting action (751
>>> bytes) to leveldb took 60.489282ms
>>> I1103 01:40:55.870177 29109 replica.cpp:708] Persisted action APPEND at
>>> position 7
>>> I1103 01:40:55.871421 29109 replica.cpp:691] Replica received learned
>>> notice for position 7 from @0.0.0.0:0
>>> I1103 01:40:55.976619 29109 leveldb.cpp:341] Persisting action (753
>>> bytes) to leveldb took 105.201942ms
>>> I1103 01:40:55.976718 29109 replica.cpp:708] Persisted action APPEND at
>>> position 7
>>> I1103 01:40:55.979708 29109 registrar.cpp:506] Successfully updated the
>>> registry in 266.301952ms
>>> I1103 01:40:55.979878 29103 log.cpp:596] Attempting to truncate the log
>>> to 7
>>> I1103 01:40:55.979991 29106 coordinator.cpp:348] Coordinator attempting
>>> to write TRUNCATE action at position 8
>>> I1103 01:40:55.980255 29112 master.cpp:5683] Updating unavailability of
>>> agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@
>>> 172.17.0.2:58302 (maintenance-host), starting at 2449.0107815972weeks
>>> I1103 01:40:55.980481 29112 master.cpp:5683] Updating unavailability of
>>> agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@
>>> 172.17.0.2:58302 (maintenance-host-2), starting at 2449.0107815972weeks
>>> I1103 01:40:55.980818 29097 replica.cpp:537] Replica received write
>>> request for position 8 from __req_res__(2521)@172.17.0.2:58302
>>> I1103 01:40:55.981092 29103 hierarchical.cpp:1694] No allocations
>>> performed
>>> I1103 01:40:55.981150 29103 hierarchical.cpp:1309] Performed allocation
>>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 in 199516ns
>>> I1103 01:40:55.981317 29103 hierarchical.cpp:1694] No allocations
>>> performed
>>> I1103 01:40:55.981367 29103 hierarchical.cpp:1309] Performed allocation
>>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 144365ns
>>> I1103 01:40:55.982864 29079 scheduler.cpp:176] Version: 1.2.0
>>> I1103 01:40:55.983412 29112 scheduler.cpp:469] New master detected at
>>> master@172.17.0.2:58302
>>> I1103 01:40:55.983438 29112 scheduler.cpp:478] Waiting for 0ns before
>>> initiating a re-(connection) attempt with the master
>>> I1103 01:40:55.984688 29098 scheduler.cpp:353] Connected with the master
>>> at http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:55.985872 29109 scheduler.cpp:235] Sending SUBSCRIBE call to
>>> http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:55.986587 29099 process.cpp:3570] Handling HTTP event for process
>>> 'master' with path: '/master/api/v1/scheduler'
>>> I1103 01:40:55.987659 29108 http.cpp:391] HTTP POST for
>>> /master/api/v1/scheduler from 172.17.0.2:45771
>>> I1103 01:40:55.988019 29108 master.cpp:2329] Received subscription
>>> request for HTTP framework 'default'
>>> I1103 01:40:55.988088 29108 master.cpp:2069] Authorizing framework
>>> principal 'test-principal' to receive offers for role '*'
>>> I1103 01:40:55.988632 29099 master.cpp:2427] Subscribing framework
>>> 'default' with checkpointing disabled and capabilities [ ]
>>> I1103 01:40:55.989305 29105 hierarchical.cpp:275] Added framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:55.989459 29099 master.hpp:2161] Sending heartbeat to
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:55.990151 29108 scheduler.cpp:675] Enqueuing event
>>> SUBSCRIBED received from http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:55.990648 29108 scheduler.cpp:675] Enqueuing event HEARTBEAT received
>>> from http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:55.991364 29105 hierarchical.cpp:1286] Performed allocation for 2
>>> agents in 2.044414ms
>>> I1103 01:40:55.992657 29100 master.cpp:6571] Sending 2 offers to
>>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
>>> I1103 01:40:55.994067 29100 master.cpp:6661] Sending 2 inverse offers to
>>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
>>> I1103 01:40:55.995620 29101 scheduler.cpp:675] Enqueuing event OFFERS
>>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:55.996536 29101 scheduler.cpp:675] Enqueuing event INVERSE_OFFERS
>>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:56.060276 29097 leveldb.cpp:341] Persisting action (16 bytes) to
>>> leveldb took 79.452553ms
>>> I1103 01:40:56.060681 29097 replica.cpp:708] Persisted action TRUNCATE
>>> at position 8
>>> I1103 01:40:56.061957 29102 replica.cpp:691] Replica received learned
>>> notice for position 8 from @0.0.0.0:0
>>> I1103 01:40:56.118927 29102 leveldb.cpp:341] Persisting action (18
>>> bytes) to leveldb took 56.925572ms
>>> I1103 01:40:56.119022 29102 leveldb.cpp:399] Deleting ~2 keys from
>>> leveldb took 51598ns
>>> I1103 01:40:56.119046 29102 replica.cpp:708] Persisted action TRUNCATE
>>> at position 8
>>> I1103 01:40:56.120704 29099 hierarchical.cpp:1694] No allocations
>>> performed
>>> I1103 01:40:56.121099 29099 hierarchical.cpp:1286] Performed allocation
>>> for 2 agents in 610047ns
>>> I1103 01:40:56.122525 29097 scheduler.cpp:235] Sending ACCEPT call to
>>> http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103> 01:40:56.123138
>>> 29097 scheduler.cpp:235] Sending ACCEPT call to
>>> http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:56.124460 29098 process.cpp:3570] Handling HTTP event for process
>>> 'master' with path: '/master/api/v1/scheduler'
>>> I1103 01:40:56.124822 29098 process.cpp:3570] Handling HTTP event for
>>> process 'master' with path: '/master/api/v1/scheduler'
>>> I1103 01:40:56.126590 29098 http.cpp:391] HTTP POST for
>>> /master/api/v1/scheduler from 172.17.0.2:45772
>>> I1103 01:40:56.127682 29098 master.cpp:3581] Processing ACCEPT call for
>>> offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O0 ] on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302
>>> (maintenance-host) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> (default)
>>> I1103 01:40:56.127856 29098 master.cpp:3173] Authorizing framework
>>> principal 'test-principal' to launch task 19b41cf9-ba0f-472e-8cae-5ea705
>>> 74b1b4
>>> I1103 01:40:56.128916 29098 http.cpp:391] HTTP POST for
>>> /master/api/v1/scheduler from 172.17.0.2:45772
>>> I1103 01:40:56.129622 29098 master.cpp:3581] Processing ACCEPT call for
>>> offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O1 ] on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>>> (maintenance-host-2) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> (default)
>>> I1103 01:40:56.129730 29098 master.cpp:3173] Authorizing framework
>>> principal 'test-principal' to launch task 846a812b-b562-4782-b0fd-9e760a
>>> 52306e
>>> W1103 01:40:56.131443 29098 validation.cpp:920] Executor 'executor-1'
>>> for task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less CPUs (None)
>>> than the minimum required (0.01). Please update your executor, as this will
>>> be mandatory in future releases.
>>> W1103 01:40:56.131489 29098 validation.cpp:932] Executor 'executor-1'
>>> for task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less memory (None)
>>> than the minimum required (32MB). Please update your executor, as this will
>>> be mandatory in future releases.
>>> I1103 01:40:56.132071 29098 master.cpp:8334] Adding task
>>> 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 with resources cpus(*):2;
>>> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 (maintenance-host)
>>> I1103 01:40:56.132442 29098 master.cpp:4230] Launching task
>>> 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) with resources
>>> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302
>>> (maintenance-host)
>>> W1103 01:40:56.133824 29098 validation.cpp:920] Executor 'executor-2'
>>> for task '846a812b-b562-4782-b0fd-9e760a52306e' uses less CPUs (None)
>>> than the minimum required (0.01). Please update your executor, as this will
>>> be mandatory in future releases.
>>> W1103 01:40:56.133858 29098 validation.cpp:932] Executor 'executor-2'
>>> for task '846a812b-b562-4782-b0fd-9e760a52306e' uses less memory (None)
>>> than the minimum required (32MB). Please update your executor, as this will
>>> be mandatory in future releases.
>>> I1103 01:40:56.134299 29097 slave.cpp:1547] Got assigned task
>>> '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.134332 29098 master.cpp:8334] Adding task
>>> 846a812b-b562-4782-b0fd-9e760a52306e with resources cpus(*):2;
>>> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2)
>>> I1103 01:40:56.134572 29098 master.cpp:4230] Launching task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) with resources
>>> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>>> (maintenance-host-2)
>>> I1103 01:40:56.135084 29097 slave.cpp:1709] Launching task
>>> '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.135145 29098 slave.cpp:1547] Got assigned task
>>> '846a812b-b562-4782-b0fd-9e760a52306e' for framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.135660 29098 slave.cpp:1709] Launching task
>>> '846a812b-b562-4782-b0fd-9e760a52306e' for framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.135871 29097 paths.cpp:536] Trying to chown
>>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_B4KX1H/slav
>>> es/3167a687-904b-4b57-bc0f-91b67dc7e41d-S0/frameworks/3167a6
>>> 87-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-1/
>>> runs/fbb81e53-b134-4196-9385-791612794545' to user 'mesos'
>>> I1103 01:40:56.138550 29098 paths.cpp:536] Trying to chown
>>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slav
>>> es/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/3167a6
>>> 87-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-2/
>>> runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4' to user 'mesos'
>>> I1103 01:40:56.144282 29098 slave.cpp:6307] Launching executor
>>> 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> with resources {} in work directory '/tmp/MasterMaintenanceTest_In
>>> verseOffersFilters_xHOK8S/slaves/3167a687-904b-4b57-bc0f-91b
>>> 67dc7e41d-S1/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d
>>> -0000/executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
>>> I1103 01:40:56.144282 29097 slave.cpp:6307] Launching executor
>>> 'executor-1' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> with resources {} in work directory '/tmp/MasterMaintenanceTest_In
>>> verseOffersFilters_B4KX1H/slaves/3167a687-904b-4b57-bc0f-91b
>>> 67dc7e41d-S0/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d
>>> -0000/executors/executor-1/runs/fbb81e53-b134-4196-9385-791612794545'
>>> F1103 01:40:56.144891 29097 owned.hpp:110] Check failed: 'get()' Must be
>>> non NULL
>>> *** Check failure stack trace: ***
>>> I1103 01:40:56.147233 29098 exec.cpp:162] Version: 1.2.0
>>> I1103 01:40:56.147598 29112 exec.cpp:212] Executor started at:
>>> executor(60)@172.17.0.2:58302 with pid 29079
>>> I1103 01:40:56.147956 29098 slave.cpp:2031] Queued task
>>> '846a812b-b562-4782-b0fd-9e760a52306e' for executor 'executor-2' of
>>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.148043 29098 slave.cpp:868] Successfully attached file
>>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slav
>>> es/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/3167a6
>>> 87-904b-4b57-bc0f-91b67dc7e41d-0000/executors/executor-2/
>>> runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
>>> I1103 01:40:56.148277 29098 slave.cpp:3305] Got registration for
>>> executor 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> from executor(60)@172.17.0.2:58302
>>> I1103 01:40:56.148833 29105 exec.cpp:237] Executor registered on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
>>> I1103 01:40:56.148895 29105 exec.cpp:249] Executor::registered took
>>> 29894ns
>>> I1103 01:40:56.149114 29098 slave.cpp:2247] Sending queued task
>>> '846a812b-b562-4782-b0fd-9e760a52306e' to executor 'executor-2' of
>>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 at executor(60)@
>>> 172.17.0.2:58302
>>> I1103 01:40:56.149495 29106 exec.cpp:324] Executor asked to run task
>>> '846a812b-b562-4782-b0fd-9e760a52306e'
>>> I1103 01:40:56.149608 29106 exec.cpp:333] Executor::launchTask took
>>> 98162ns
>>> I1103 01:40:56.149705 29106 exec.cpp:550] Executor sending status update
>>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.149987 29106 slave.cpp:3740] Handling status update
>>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from executor(60)@
>>> 172.17.0.2:58302
>>> I1103 01:40:56.150545 29098 status_update_manager.cpp:323] Received
>>> status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
>>> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.150606 29098 status_update_manager.cpp:500] Creating
>>> StatusUpdate stream for task 846a812b-b562-4782-b0fd-9e760a52306e of
>>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.151116 29098 status_update_manager.cpp:377] Forwarding
>>> update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for
>>> task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to the agent
>>> @ 0x2b56852902fd google::LogMessage::Fail()
>>> I1103 01:40:56.151470 29101 slave.cpp:4169] Forwarding the update
>>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to master@172.17.0.2:58302
>>> I1103 01:40:56.151757 29101 slave.cpp:4063] Status update manager
>>> successfully handled status update TASK_RUNNING (UUID:
>>> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.151860 29101 slave.cpp:4079] Sending acknowledgement for
>>> status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
>>> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to executor(60)@
>>> 172.17.0.2:58302
>>> I1103 01:40:56.151954 29111 master.cpp:5757] Status update TASK_RUNNING
>>> (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>>> (maintenance-host-2)
>>> I1103 01:40:56.152041 29111 master.cpp:5819] Forwarding status update
>>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.152123 29100 exec.cpp:373] Executor received status
>>> update acknowledgement 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.152546 29111 master.cpp:7712] Updating the state of task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (latest state: TASK_RUNNING,
>>> status update state: TASK_RUNNING)
>>> I1103 01:40:56.153616 29103 scheduler.cpp:675] Enqueuing event UPDATE
>>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>>> @ 0x2b568528f6de google::LogMessage::SendToLog()
>>> I1103 01:40:56.164335 29109 scheduler.cpp:235] Sending ACKNOWLEDGE call
>>> to http://172.17.0.2:58302/master/api/v1/scheduler
>>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>>> 01:40:56.165396 29109 process.cpp:3570] Handling HTTP event for process
>>> 'master' with path: '/master/api/v1/scheduler'
>>> I1103 01:40:56.167019 29109 http.cpp:391] HTTP POST for
>>> /master/api/v1/scheduler from 172.17.0.2:45772
>>> I1103 01:40:56.167212 29109 master.cpp:4867] Processing ACKNOWLEDGE call
>>> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) on agent
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
>>> I1103 01:40:56.168145 29110 status_update_manager.cpp:395] Received
>>> status update acknowledgement (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
>>> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> I1103 01:40:56.168666 29110 slave.cpp:3022] Status update manager
>>> successfully handled status update acknowledgement (UUID:
>>> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>>> @ 0x2b568528ffbd google::LogMessage::Flush()
>>> @ 0x2b5685293438 google::LogMessageFatal::~LogMessageFatal()
>>> @ 0xcb1fc4 google::CheckNotNull<>()
>>> @ 0xc93ba6 process::Owned<>::operator->()
>>> @ 0xc9100c mesos::internal::tests::TestCo
>>> ntainerizer::_launch()
>>> @ 0xc98182 _ZN7testing8internal12InvokeHe
>>> lperIN7process6FutureIbEESt5tupleIJRKN5mesos11ContainerIDERK
>>> 6OptionINS6_8TaskInfoEERKNS6_12ExecutorInfoERKSsRKSA_ISsERKN
>>> S6_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISI_SsEEEbEEE1
>>> 2InvokeMethodINS6_8internal5tests17TestContainerizerEMS14_FS
>>> 4_S9_SE_SH_SJ_SM_SP_SY_bEEES4_PT_T0_RKSZ_
>>> @ 0xc98014 _ZNK7testing8internal18InvokeM
>>> ethodActionIN5mesos8internal5tests17TestContainerizerEMS5_FN
>>> 7process6FutureIbEERKNS2_11ContainerIDERK6OptionINS2_8TaskIn
>>> foEERKNS2_12ExecutorInfoERKSsRKSC_ISsERKNS2_7SlaveIDERKSt3ma
>>> pISsSsSt4lessISsESaISt4pairISK_SsEEEbEE7PerformIS8_St5tupleI
>>> JSB_SG_SJ_SL_SO_SR_S10_bEEEET_RKT0_
>>> @ 0xc97fb6 _ZN7testing17PolymorphicAction
>>> INS_8internal18InvokeMethodActionIN5mesos8internal5tests17Te
>>> stContainerizerEMS6_FN7process6FutureIbEERKNS3_11ContainerID
>>> ERK6OptionINS3_8TaskInfoEERKNS3_12ExecutorInfoERKSsRKSD_ISsE
>>> RKNS3_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISL_SsEEEbE
>>> EEE15MonomorphicImplIFS9_SC_SH_SK_SM_SP_SS_S11_bEE7PerformER
>>> KSt5tupleIJSC_SH_SK_SM_SP_SS_S11_bEE
>>> @ 0xb51d58 _ZNK7testing6ActionIFN7process
>>> 6FutureIbEERKN5mesos11ContainerIDERK6OptionINS4_8TaskInfoEER
>>> KNS4_12ExecutorInfoERKSsRKS8_ISsERKNS4_7SlaveIDERKSt3mapISsS
>>> sSt4lessISsESaISt4pairISG_SsEEEbEE7PerformERKSt5tupleIJS7_SC
>>> _SF_SH_SK_SN_SW_bEE
>>> @ 0xb51b6a testing::internal::ActionResul
>>> tHolder<>::PerformAction<>()
>>> @ 0xb4ca1c testing::internal::FunctionMoc
>>> kerBase<>::UntypedPerformAction()
>>> @ 0x1daa619 testing::internal::UntypedFunc
>>> tionMockerBase::UntypedInvokeWith()
>>> @ 0xb4995b _ZN7testing8internal18Function
>>> MockerBaseIFN7process6FutureIbEERKN5mesos11ContainerIDERK6Op
>>> tionINS5_8TaskInfoEERKNS5_12ExecutorInfoERKSsRKS9_ISsERKNS5_
>>> 7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISH_SsEEEbEE10Inv
>>> okeWithERKSt5tupleIJS8_SD_SG_SI_SL_SO_SX_bEE
>>> @ 0xb49916 testing::internal::FunctionMocker<>::Invoke()
>>> @ 0xc9712f mesos::internal::tests::TestCo
>>> ntainerizer::launch()
>>> @ 0x2b5684298b9b mesos::internal::slave::Framew
>>> ork::launchExecutor()
>>> @ 0x2b568429607e mesos::internal::slave::Slave::_run()
>>> @ 0x2b56843265a0 _ZZN7process8dispatchIN5mesos8
>>> internal5slave5SlaveERKNS_6FutureIbEERKNS1_13FrameworkInfoER
>>> KNS1_12ExecutorInfoERK6OptionINS1_8TaskInfoEERKSF_INS1_13Tas
>>> kGroupInfoEES6_S9_SC_SH_SL_EEvRKNS_3PIDIT_EEMSP_FvT0_T1_T2_
>>> T3_T4_ET5_T6_T7_T8_T9_ENKUlPNS_11ProcessBaseEE_clES16_
>>> @ 0x2b5684326062 _ZNSt17_Function_handlerIFvPN7
>>> process11ProcessBaseEEZNS0_8dispatchIN5mesos8internal5slave5
>>> SlaveERKNS0_6FutureIbEERKNS5_13FrameworkInfoERKNS5_12Executo
>>> rInfoERK6OptionINS5_8TaskInfoEERKSJ_INS5_13TaskGroupInfoEESA
>>> _SD_SG_SL_SP_EEvRKNS0_3PIDIT_EEMST_FvT0_T1_T2_T3_T4_ET5_T6_T
>>> 7_T8_T9_EUlS2_E_E9_M_invokeERKSt9_Any_dataS2_
>>> @ 0x2b56851b2dd8 std::function<>::operator()()
>>> @ 0x2b5685199f74 process::ProcessBase::visit()
>>> @ 0x2b56851f84ee process::DispatchEvent::visit()
>>> @ 0x8dabe1 process::ProcessBase::serve()
>>> @ 0x2b5685197c64 process::ProcessManager::resume()
>>> @ 0x2b56851a28cc process::ProcessManager::init_
>>> threads()::$_0::operator()()
>>> @ 0x2b56851a27d5 _ZNSt12_Bind_simpleIFZN7proces
>>> s14ProcessManager12init_threadsEvE3$_0vEE9_M_invokeIJEEEvSt1
>>> 2_Index_tupleIJXspT_EEE
>>> @ 0x2b56851a27a5 std::_Bind_simple<>::operator()()
>>> @ 0x2b56851a277c std::thread::_Impl<>::_M_run()
>>> @ 0x2b5686d10a60 (unknown)
>>> @ 0x2b5687487184 start_thread
>>> make[4]: *** [check-local] Aborted
>>> make[4]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
>>> make[3]: *** [check-am] Error 2
>>> make[3]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
>>> make[2]: *** [check] Error 2
>>> make[2]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
>>> make[1]: *** [check-recursive] Error 1
>>> make[1]: Leaving directory `/mesos/mesos-1.2.0/_build'
>>> make: *** [distcheck] Error 1
>>> + docker rmi mesos-1478134041-23345
>>> Untagged: mesos-1478134041-23345:latest
>>> Deleted: sha256:74596d8e8b76315d1115980623d7e5af9150c10d54203c726895a
>>> a5cac6da79e
>>> Deleted: sha256:7a0b0a4cf4ac2c9897b51052cb68965f0b4e8a305ff6db16b7d53
>>> 0709b6e406c
>>> Deleted: sha256:567f543714141c74573998d4209337287ffbe0e8ccd9ee8e4b044
>>> cc6e872c0e5
>>> Deleted: sha256:fbd8be03b0d3f1c70dea858388d951374325b608d3e56c6b51fd1
>>> 02b193afed2
>>> Deleted: sha256:7c7aee38843610c581d675c5620d8981f65d7fda18fc0fa05a264
>>> fb510dd927c
>>> Deleted: sha256:654d69e9689193c66be28a0fff218d21a5a119fbeecdc26a8a4cf
>>> 34b3c2cd104
>>> Deleted: sha256:4f5e0007d3903ea948c9fa13032fad2b1244dc0d4dbbee0673cd3
>>> 470c8f3c24a
>>> Deleted: sha256:38239a48898a69f2bf752d732a101a22038fc2d8e023b6c789987
>>> daa2332ee1b
>>> Deleted: sha256:329c7b62b50ac8578b3499838a1f12435013f0b4f429da2efac3d
>>> 76673800950
>>> Deleted: sha256:45dc8a92ffad997c6e48598af8e4665ff06cbb6d6316351de7103
>>> 17f420367a2
>>> Deleted: sha256:bc4e9bad738f327a22d9bb1204cbb3787b6b04fcda850497d7a6f
>>> d54b1d9b9f3
>>> Deleted: sha256:708772f854104da0e8b60eb0a9fb12c5cd85fca258ed74289e0ca
>>> dcab12bb2f9
>>> Deleted: sha256:5498b1ca089ca60b79713f6a57bc63036a89079ae1afff2ec21d4
>>> 42d5fa531f1
>>> Deleted: sha256:7830805a402ac02a943fc9dbac0ff1dd46c4cfd67af2fb4e36b9d
>>> 2e43dd7a4d5
>>> Deleted: sha256:354a09c0839db0533726726c863f23138aad1656c38ee8f3c16b0
>>> 4b1766a0109
>>> Deleted: sha256:73999f055e575a9767df52d22b67980d2b536b8ac1eeda0e9af1e
>>> a623fc939b9
>>> Deleted: sha256:c915f730bb38299ca517a3f295fcdd21e0078d959381ae5a9aff0
>>> f39b4748bd5
>>> Deleted: sha256:ca688ce3b2900d35c52872f38fd4af8530b780bb362e6e1a42687
>>> d4f5e4082e8
>>> Deleted: sha256:d2c33d1bb8213d5ed68c888aa09a65370a36d6ab4311704d2f106
>>> e7ef94cb6fc
>>> Deleted: sha256:67e4c09745aa1fd979f24bfae0812811bd44377dafd43130c0c27
>>> 69e820d252a
>>> Deleted: sha256:958624053829cf5b33921f84ad226d1835a271befa6856f591fc0
>>> b4e1981c349
>>> Deleted: sha256:a59579b1b2fd24a0570169c6a97d2ba86c88f44a9ac159c792f37
>>> 71c9d3585a7
>>> Deleted: sha256:4edbd39dc9e6dcb72d14b8e207b6705bace589ce1dc8cfe4525ff
>>> ac3d216f471
>>> Deleted: sha256:92fc49f6ea70503f089441fdcdfdcf8ebc3875060deefaf5f355c
>>> 4a9b45314e8
>>> Build step 'Execute shell' marked build as failure
>>>
>>
>>
>
Re: Build failed in Jenkins: Mesos » autotools,clang,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,ubuntu:14.04,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2873
Posted by Benjamin Mahler <bm...@apache.org>.
https://issues.apache.org/jira/browse/MESOS-6544
https://issues.apache.org/jira/browse/MESOS-6545
On Thu, Nov 3, 2016 at 4:56 PM, Benjamin Mahler <bm...@apache.org> wrote:
> This test uses two mock executors across two agents, which means the
> agents may concurrently call into the TestContainerizer to launch the
> container. The TestContainerizer is not a Process and so we may
> concurrently modify its data structures, I suspect that the following
> executes concurrently and leads to the crash:
>
> https://github.com/apache/mesos/blob/44242c058158727ce013bd51764368
> f5e120ee75/src/tests/containerizer.cpp#L131
>
> On Wed, Nov 2, 2016 at 6:41 PM, Apache Jenkins Server <
> jenkins@builds.apache.org> wrote:
>
>> See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,COM
>> PILER=clang,CONFIGURATION=--verbose%20--enable-libevent%20
>> --enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=ubunt
>> u%3A14.04,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-us1)&&(
>> !ubuntu-6)/2873/changes>
>>
>> Changes:
>>
>> [josephwu] Fixed some markdown whitespace style.
>>
>> [josephwu] Fixed a typo in `gpu-support.md`.
>>
>> [josephwu] Fixed FD leak due to exiting early in Mesos fetcher.
>>
>> ------------------------------------------
>> [...truncated 78174 lines...]
>> I1103 01:40:55.530350 29098 slave.cpp:974] Authenticating with master
>> master@172.17.0.2:58302
>> I1103 01:40:55.530432 29098 slave.cpp:985] Using default CRAM-MD5
>> authenticatee
>> I1103 01:40:55.530627 29098 slave.cpp:947] Detecting new master
>> I1103 01:40:55.530675 29108 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I1103 01:40:55.530743 29098 slave.cpp:5587] Received oversubscribable
>> resources {} from the resource estimator
>> I1103 01:40:55.530961 29099 master.cpp:6742] Authenticating slave(150)@
>> 172.17.0.2:58302
>> I1103 01:40:55.531070 29112 authenticator.cpp:414] Starting
>> authentication session for crammd5-authenticatee(357)@172.17.0.2:58302
>> I1103 01:40:55.531328 29106 authenticator.cpp:98] Creating new server
>> SASL connection
>> I1103 01:40:55.531561 29108 authenticatee.cpp:213] Received SASL
>> authentication mechanisms: CRAM-MD5
>> I1103 01:40:55.531604 29108 authenticatee.cpp:239] Attempting to
>> authenticate with mechanism 'CRAM-MD5'
>> I1103 01:40:55.531713 29101 authenticator.cpp:204] Received SASL
>> authentication start
>> I1103 01:40:55.531805 29101 authenticator.cpp:326] Authentication
>> requires more steps
>> I1103 01:40:55.531921 29108 authenticatee.cpp:259] Received SASL
>> authentication step
>> I1103 01:40:55.532120 29101 authenticator.cpp:232] Received SASL
>> authentication step
>> I1103 01:40:55.532155 29101 auxprop.cpp:109] Request to lookup properties
>> for user: 'test-principal' realm: '3a1c598ce334' server FQDN:
>> '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
>> I1103 01:40:55.532179 29101 auxprop.cpp:181] Looking up auxiliary
>> property '*userPassword'
>> I1103 01:40:55.532233 29101 auxprop.cpp:181] Looking up auxiliary
>> property '*cmusaslsecretCRAM-MD5'
>> I1103 01:40:55.532266 29101 auxprop.cpp:109] Request to lookup properties
>> for user: 'test-principal' realm: '3a1c598ce334' server FQDN:
>> '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
>> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
>> I1103 01:40:55.532289 29101 auxprop.cpp:131] Skipping auxiliary property
>> '*userPassword' since SASL_AUXPROP_AUTHZID == true
>> I1103 01:40:55.532305 29101 auxprop.cpp:131] Skipping auxiliary property
>> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
>> I1103 01:40:55.532335 29101 authenticator.cpp:318] Authentication success
>> I1103 01:40:55.532413 29110 authenticatee.cpp:299] Authentication success
>> I1103 01:40:55.532467 29108 master.cpp:6772] Successfully authenticated
>> principal 'test-principal' at slave(150)@172.17.0.2:58302
>> I1103 01:40:55.532536 29111 authenticator.cpp:432] Authentication session
>> cleanup for crammd5-authenticatee(357)@172.17.0.2:58302
>> I1103 01:40:55.532755 29098 slave.cpp:1069] Successfully authenticated
>> with master master@172.17.0.2:58302
>> I1103 01:40:55.532997 29098 slave.cpp:1483] Will retry registration in
>> 12.590371ms if necessary
>> I1103 01:40:55.533179 29108 master.cpp:5151] Registering agent at
>> slave(150)@172.17.0.2:58302 (maintenance-host-2) with id
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
>> I1103 01:40:55.533572 29112 registrar.cpp:461] Applied 1 operations in
>> 94467ns; attempting to update the registry
>> I1103 01:40:55.546341 29107 slave.cpp:1483] Will retry registration in
>> 36.501523ms if necessary
>> I1103 01:40:55.546461 29099 master.cpp:5139] Ignoring register agent
>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>> admission is already in progress
>> I1103 01:40:55.565403 29097 leveldb.cpp:341] Persisting action (16 bytes)
>> to leveldb took 48.099208ms
>> I1103 01:40:55.565495 29097 replica.cpp:708] Persisted action TRUNCATE at
>> position 4
>> I1103 01:40:55.566788 29097 replica.cpp:691] Replica received learned
>> notice for position 4 from @0.0.0.0:0
>> I1103 01:40:55.583937 29101 slave.cpp:1483] Will retry registration in
>> 26.127711ms if necessary
>> I1103 01:40:55.584123 29112 master.cpp:5139] Ignoring register agent
>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>> admission is already in progress
>> I1103 01:40:55.609695 29097 leveldb.cpp:341] Persisting action (18 bytes)
>> to leveldb took 42.905697ms
>> I1103 01:40:55.609860 29097 leveldb.cpp:399] Deleting ~2 keys from
>> leveldb took 96623ns
>> I1103 01:40:55.609899 29097 replica.cpp:708] Persisted action TRUNCATE at
>> position 4
>> I1103 01:40:55.611063 29106 log.cpp:577] Attempting to append 513 bytes
>> to the log
>> I1103 01:40:55.611229 29097 coordinator.cpp:348] Coordinator attempting
>> to write APPEND action at position 5
>> I1103 01:40:55.611498 29100 slave.cpp:1483] Will retry registration in
>> 85.55417ms if necessary
>> I1103 01:40:55.612069 29105 master.cpp:5139] Ignoring register agent
>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>> admission is already in progress
>> I1103 01:40:55.612313 29100 replica.cpp:537] Replica received write
>> request for position 5 from __req_res__(2518)@172.17.0.2:58302
>> I1103 01:40:55.657845 29100 leveldb.cpp:341] Persisting action (532
>> bytes) to leveldb took 45.517822ms
>> I1103 01:40:55.657938 29100 replica.cpp:708] Persisted action APPEND at
>> position 5
>> I1103 01:40:55.658681 29103 replica.cpp:691] Replica received learned
>> notice for position 5 from @0.0.0.0:0
>> I1103 01:40:55.698050 29112 slave.cpp:1483] Will retry registration in
>> 100.32384ms if necessary
>> I1103 01:40:55.698319 29101 master.cpp:5139] Ignoring register agent
>> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
>> admission is already in progress
>> I1103 01:40:55.705086 29103 leveldb.cpp:341] Persisting action (534
>> bytes) to leveldb took 46.384557ms
>> I1103 01:40:55.705157 29103 replica.cpp:708] Persisted action APPEND at
>> position 5
>> I1103 01:40:55.707480 29098 registrar.cpp:506] Successfully updated the
>> registry in 173824us
>> I1103 01:40:55.707741 29100 log.cpp:596] Attempting to truncate the log
>> to 5
>> I1103 01:40:55.708029 29110 coordinator.cpp:348] Coordinator attempting
>> to write TRUNCATE action at position 6
>> I1103 01:40:55.708501 29097 slave.cpp:4251] Received ping from
>> slave-observer(151)@172.17.0.2:58302
>> I1103 01:40:55.708528 29112 master.cpp:5222] Registered agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>> (maintenance-host-2) with cpus(*):2; mem(*):1024; disk(*):1024;
>> ports(*):[31000-32000]
>> I1103 01:40:55.708796 29097 slave.cpp:1115] Registered with master
>> master@172.17.0.2:58302; given agent ID 3167a687-904b-4b57-bc0f-91b67d
>> c7e41d-S1
>> I1103 01:40:55.708822 29097 fetcher.cpp:86] Clearing fetcher cache
>> I1103 01:40:55.708889 29103 replica.cpp:537] Replica received write
>> request for position 6 from __req_res__(2519)@172.17.0.2:58302
>> I1103 01:40:55.708935 29100 hierarchical.cpp:485] Added agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) with
>> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
>> I1103 01:40:55.709065 29105 status_update_manager.cpp:184] Resuming
>> sending status updates
>> I1103 01:40:55.709139 29100 hierarchical.cpp:1694] No allocations
>> performed
>> I1103 01:40:55.709204 29100 hierarchical.cpp:1309] Performed allocation
>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 229932ns
>> I1103 01:40:55.709316 29097 slave.cpp:1138] Checkpointing SlaveInfo to
>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/meta
>> /slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/slave.info'
>> I1103 01:40:55.709650 29097 slave.cpp:1175] Forwarding total
>> oversubscribed resources {}
>> I1103 01:40:55.709787 29097 master.cpp:5621] Received update of agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>> (maintenance-host-2) with total oversubscribed resources {}
>> I1103 01:40:55.710000 29108 hierarchical.cpp:555] Agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) updated
>> with oversubscribed resources {} (total: cpus(*):2; mem(*):1024;
>> disk(*):1024; ports(*):[31000-32000], allocated: {})
>> I1103 01:40:55.710144 29108 hierarchical.cpp:1694] No allocations
>> performed
>> I1103 01:40:55.710199 29108 hierarchical.cpp:1309] Performed allocation
>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 157491ns
>> I1103 01:40:55.711632 29105 process.cpp:3570] Handling HTTP event for
>> process 'master' with path: '/master/maintenance/schedule'
>> I1103 01:40:55.712441 29111 http.cpp:391] HTTP POST for
>> /master/maintenance/schedule from 172.17.0.2:45755
>> I1103 01:40:55.713325 29110 registrar.cpp:461] Applied 1 operations in
>> 165622ns; attempting to update the registry
>> I1103 01:40:55.756808 29103 leveldb.cpp:341] Persisting action (16 bytes)
>> to leveldb took 47.899715ms
>> I1103 01:40:55.756888 29103 replica.cpp:708] Persisted action TRUNCATE at
>> position 6
>> I1103 01:40:55.758083 29103 replica.cpp:691] Replica received learned
>> notice for position 6 from @0.0.0.0:0
>> I1103 01:40:55.807322 29103 leveldb.cpp:341] Persisting action (18 bytes)
>> to leveldb took 49.23844ms
>> I1103 01:40:55.807481 29103 leveldb.cpp:399] Deleting ~2 keys from
>> leveldb took 90240ns
>> I1103 01:40:55.807517 29103 replica.cpp:708] Persisted action TRUNCATE at
>> position 6
>> I1103 01:40:55.808619 29101 log.cpp:577] Attempting to append 732 bytes
>> to the log
>> I1103 01:40:55.808737 29097 coordinator.cpp:348] Coordinator attempting
>> to write APPEND action at position 7
>> I1103 01:40:55.809571 29109 replica.cpp:537] Replica received write
>> request for position 7 from __req_res__(2520)@172.17.0.2:58302
>> I1103 01:40:55.810784 29098 hierarchical.cpp:1694] No allocations
>> performed
>> I1103 01:40:55.810860 29098 hierarchical.cpp:1286] Performed allocation
>> for 2 agents in 282405ns
>> I1103 01:40:55.870082 29109 leveldb.cpp:341] Persisting action (751
>> bytes) to leveldb took 60.489282ms
>> I1103 01:40:55.870177 29109 replica.cpp:708] Persisted action APPEND at
>> position 7
>> I1103 01:40:55.871421 29109 replica.cpp:691] Replica received learned
>> notice for position 7 from @0.0.0.0:0
>> I1103 01:40:55.976619 29109 leveldb.cpp:341] Persisting action (753
>> bytes) to leveldb took 105.201942ms
>> I1103 01:40:55.976718 29109 replica.cpp:708] Persisted action APPEND at
>> position 7
>> I1103 01:40:55.979708 29109 registrar.cpp:506] Successfully updated the
>> registry in 266.301952ms
>> I1103 01:40:55.979878 29103 log.cpp:596] Attempting to truncate the log
>> to 7
>> I1103 01:40:55.979991 29106 coordinator.cpp:348] Coordinator attempting
>> to write TRUNCATE action at position 8
>> I1103 01:40:55.980255 29112 master.cpp:5683] Updating unavailability of
>> agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@
>> 172.17.0.2:58302 (maintenance-host), starting at 2449.0107815972weeks
>> I1103 01:40:55.980481 29112 master.cpp:5683] Updating unavailability of
>> agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@
>> 172.17.0.2:58302 (maintenance-host-2), starting at 2449.0107815972weeks
>> I1103 01:40:55.980818 29097 replica.cpp:537] Replica received write
>> request for position 8 from __req_res__(2521)@172.17.0.2:58302
>> I1103 01:40:55.981092 29103 hierarchical.cpp:1694] No allocations
>> performed
>> I1103 01:40:55.981150 29103 hierarchical.cpp:1309] Performed allocation
>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 in 199516ns
>> I1103 01:40:55.981317 29103 hierarchical.cpp:1694] No allocations
>> performed
>> I1103 01:40:55.981367 29103 hierarchical.cpp:1309] Performed allocation
>> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 144365ns
>> I1103 01:40:55.982864 29079 scheduler.cpp:176] Version: 1.2.0
>> I1103 01:40:55.983412 29112 scheduler.cpp:469] New master detected at
>> master@172.17.0.2:58302
>> I1103 01:40:55.983438 29112 scheduler.cpp:478] Waiting for 0ns before
>> initiating a re-(connection) attempt with the master
>> I1103 01:40:55.984688 29098 scheduler.cpp:353] Connected with the master
>> at http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:55.985872 29109 scheduler.cpp:235] Sending SUBSCRIBE call to
>> http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:55.986587 29099 process.cpp:3570] Handling HTTP event for process
>> 'master' with path: '/master/api/v1/scheduler'
>> I1103 01:40:55.987659 29108 http.cpp:391] HTTP POST for
>> /master/api/v1/scheduler from 172.17.0.2:45771
>> I1103 01:40:55.988019 29108 master.cpp:2329] Received subscription
>> request for HTTP framework 'default'
>> I1103 01:40:55.988088 29108 master.cpp:2069] Authorizing framework
>> principal 'test-principal' to receive offers for role '*'
>> I1103 01:40:55.988632 29099 master.cpp:2427] Subscribing framework
>> 'default' with checkpointing disabled and capabilities [ ]
>> I1103 01:40:55.989305 29105 hierarchical.cpp:275] Added framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:55.989459 29099 master.hpp:2161] Sending heartbeat to
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:55.990151 29108 scheduler.cpp:675] Enqueuing event SUBSCRIBED
>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:55.990648 29108 scheduler.cpp:675] Enqueuing event HEARTBEAT received
>> from http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:55.991364 29105 hierarchical.cpp:1286] Performed allocation for 2
>> agents in 2.044414ms
>> I1103 01:40:55.992657 29100 master.cpp:6571] Sending 2 offers to
>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
>> I1103 01:40:55.994067 29100 master.cpp:6661] Sending 2 inverse offers to
>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
>> I1103 01:40:55.995620 29101 scheduler.cpp:675] Enqueuing event OFFERS
>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:55.996536 29101 scheduler.cpp:675] Enqueuing event INVERSE_OFFERS
>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:56.060276 29097 leveldb.cpp:341] Persisting action (16 bytes) to
>> leveldb took 79.452553ms
>> I1103 01:40:56.060681 29097 replica.cpp:708] Persisted action TRUNCATE at
>> position 8
>> I1103 01:40:56.061957 29102 replica.cpp:691] Replica received learned
>> notice for position 8 from @0.0.0.0:0
>> I1103 01:40:56.118927 29102 leveldb.cpp:341] Persisting action (18 bytes)
>> to leveldb took 56.925572ms
>> I1103 01:40:56.119022 29102 leveldb.cpp:399] Deleting ~2 keys from
>> leveldb took 51598ns
>> I1103 01:40:56.119046 29102 replica.cpp:708] Persisted action TRUNCATE at
>> position 8
>> I1103 01:40:56.120704 29099 hierarchical.cpp:1694] No allocations
>> performed
>> I1103 01:40:56.121099 29099 hierarchical.cpp:1286] Performed allocation
>> for 2 agents in 610047ns
>> I1103 01:40:56.122525 29097 scheduler.cpp:235] Sending ACCEPT call to
>> http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103> 01:40:56.123138
>> 29097 scheduler.cpp:235] Sending ACCEPT call to
>> http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:56.124460 29098 process.cpp:3570] Handling HTTP event for process
>> 'master' with path: '/master/api/v1/scheduler'
>> I1103 01:40:56.124822 29098 process.cpp:3570] Handling HTTP event for
>> process 'master' with path: '/master/api/v1/scheduler'
>> I1103 01:40:56.126590 29098 http.cpp:391] HTTP POST for
>> /master/api/v1/scheduler from 172.17.0.2:45772
>> I1103 01:40:56.127682 29098 master.cpp:3581] Processing ACCEPT call for
>> offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O0 ] on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302
>> (maintenance-host) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> (default)
>> I1103 01:40:56.127856 29098 master.cpp:3173] Authorizing framework
>> principal 'test-principal' to launch task 19b41cf9-ba0f-472e-8cae-5ea705
>> 74b1b4
>> I1103 01:40:56.128916 29098 http.cpp:391] HTTP POST for
>> /master/api/v1/scheduler from 172.17.0.2:45772
>> I1103 01:40:56.129622 29098 master.cpp:3581] Processing ACCEPT call for
>> offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O1 ] on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>> (maintenance-host-2) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> (default)
>> I1103 01:40:56.129730 29098 master.cpp:3173] Authorizing framework
>> principal 'test-principal' to launch task 846a812b-b562-4782-b0fd-9e760a
>> 52306e
>> W1103 01:40:56.131443 29098 validation.cpp:920] Executor 'executor-1' for
>> task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less CPUs (None) than
>> the minimum required (0.01). Please update your executor, as this will be
>> mandatory in future releases.
>> W1103 01:40:56.131489 29098 validation.cpp:932] Executor 'executor-1' for
>> task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less memory (None) than
>> the minimum required (32MB). Please update your executor, as this will be
>> mandatory in future releases.
>> I1103 01:40:56.132071 29098 master.cpp:8334] Adding task
>> 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 with resources cpus(*):2;
>> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 (maintenance-host)
>> I1103 01:40:56.132442 29098 master.cpp:4230] Launching task
>> 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) with resources
>> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302
>> (maintenance-host)
>> W1103 01:40:56.133824 29098 validation.cpp:920] Executor 'executor-2' for
>> task '846a812b-b562-4782-b0fd-9e760a52306e' uses less CPUs (None) than
>> the minimum required (0.01). Please update your executor, as this will be
>> mandatory in future releases.
>> W1103 01:40:56.133858 29098 validation.cpp:932] Executor 'executor-2' for
>> task '846a812b-b562-4782-b0fd-9e760a52306e' uses less memory (None) than
>> the minimum required (32MB). Please update your executor, as this will be
>> mandatory in future releases.
>> I1103 01:40:56.134299 29097 slave.cpp:1547] Got assigned task
>> '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.134332 29098 master.cpp:8334] Adding task
>> 846a812b-b562-4782-b0fd-9e760a52306e with resources cpus(*):2;
>> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2)
>> I1103 01:40:56.134572 29098 master.cpp:4230] Launching task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) with resources
>> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>> (maintenance-host-2)
>> I1103 01:40:56.135084 29097 slave.cpp:1709] Launching task
>> '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.135145 29098 slave.cpp:1547] Got assigned task
>> '846a812b-b562-4782-b0fd-9e760a52306e' for framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.135660 29098 slave.cpp:1709] Launching task
>> '846a812b-b562-4782-b0fd-9e760a52306e' for framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.135871 29097 paths.cpp:536] Trying to chown
>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_B4KX1H/slav
>> es/3167a687-904b-4b57-bc0f-91b67dc7e41d-S0/frameworks/
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/
>> executor-1/runs/fbb81e53-b134-4196-9385-791612794545' to user 'mesos'
>> I1103 01:40:56.138550 29098 paths.cpp:536] Trying to chown
>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slav
>> es/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/
>> executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4' to user 'mesos'
>> I1103 01:40:56.144282 29098 slave.cpp:6307] Launching executor
>> 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 with
>> resources {} in work directory '/tmp/MasterMaintenanceTest_In
>> verseOffersFilters_xHOK8S/slaves/3167a687-904b-4b57-bc0f-
>> 91b67dc7e41d-S1/frameworks/3167a687-904b-4b57-bc0f-
>> 91b67dc7e41d-0000/executors/executor-2/runs/c7d8cc2e-b173-
>> 4dbe-b5ae-e45402c1cdf4'
>> I1103 01:40:56.144282 29097 slave.cpp:6307] Launching executor
>> 'executor-1' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 with
>> resources {} in work directory '/tmp/MasterMaintenanceTest_In
>> verseOffersFilters_B4KX1H/slaves/3167a687-904b-4b57-bc0f-
>> 91b67dc7e41d-S0/frameworks/3167a687-904b-4b57-bc0f-
>> 91b67dc7e41d-0000/executors/executor-1/runs/fbb81e53-b134-
>> 4196-9385-791612794545'
>> F1103 01:40:56.144891 29097 owned.hpp:110] Check failed: 'get()' Must be
>> non NULL
>> *** Check failure stack trace: ***
>> I1103 01:40:56.147233 29098 exec.cpp:162] Version: 1.2.0
>> I1103 01:40:56.147598 29112 exec.cpp:212] Executor started at:
>> executor(60)@172.17.0.2:58302 with pid 29079
>> I1103 01:40:56.147956 29098 slave.cpp:2031] Queued task
>> '846a812b-b562-4782-b0fd-9e760a52306e' for executor 'executor-2' of
>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.148043 29098 slave.cpp:868] Successfully attached file
>> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/slav
>> es/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/frameworks/
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/executors/
>> executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
>> I1103 01:40:56.148277 29098 slave.cpp:3305] Got registration for executor
>> 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from
>> executor(60)@172.17.0.2:58302
>> I1103 01:40:56.148833 29105 exec.cpp:237] Executor registered on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
>> I1103 01:40:56.148895 29105 exec.cpp:249] Executor::registered took
>> 29894ns
>> I1103 01:40:56.149114 29098 slave.cpp:2247] Sending queued task
>> '846a812b-b562-4782-b0fd-9e760a52306e' to executor 'executor-2' of
>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 at executor(60)@
>> 172.17.0.2:58302
>> I1103 01:40:56.149495 29106 exec.cpp:324] Executor asked to run task
>> '846a812b-b562-4782-b0fd-9e760a52306e'
>> I1103 01:40:56.149608 29106 exec.cpp:333] Executor::launchTask took
>> 98162ns
>> I1103 01:40:56.149705 29106 exec.cpp:550] Executor sending status update
>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.149987 29106 slave.cpp:3740] Handling status update
>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from executor(60)@
>> 172.17.0.2:58302
>> I1103 01:40:56.150545 29098 status_update_manager.cpp:323] Received
>> status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
>> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.150606 29098 status_update_manager.cpp:500] Creating
>> StatusUpdate stream for task 846a812b-b562-4782-b0fd-9e760a52306e of
>> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.151116 29098 status_update_manager.cpp:377] Forwarding
>> update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for
>> task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to the agent
>> @ 0x2b56852902fd google::LogMessage::Fail()
>> I1103 01:40:56.151470 29101 slave.cpp:4169] Forwarding the update
>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to master@172.17.0.2:58302
>> I1103 01:40:56.151757 29101 slave.cpp:4063] Status update manager
>> successfully handled status update TASK_RUNNING (UUID:
>> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.151860 29101 slave.cpp:4079] Sending acknowledgement for
>> status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
>> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to executor(60)@
>> 172.17.0.2:58302
>> I1103 01:40:56.151954 29111 master.cpp:5757] Status update TASK_RUNNING
>> (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
>> (maintenance-host-2)
>> I1103 01:40:56.152041 29111 master.cpp:5819] Forwarding status update
>> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.152123 29100 exec.cpp:373] Executor received status update
>> acknowledgement 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.152546 29111 master.cpp:7712] Updating the state of task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (latest state: TASK_RUNNING,
>> status update state: TASK_RUNNING)
>> I1103 01:40:56.153616 29103 scheduler.cpp:675] Enqueuing event UPDATE
>> received from http://172.17.0.2:58302/master/api/v1/scheduler
>> @ 0x2b568528f6de google::LogMessage::SendToLog()
>> I1103 01:40:56.164335 29109 scheduler.cpp:235] Sending ACKNOWLEDGE call
>> to http://172.17.0.2:58302/master/api/v1/scheduler
>> I1103 <http://172.17.0.2:58302/master/api/v1/schedulerI1103>
>> 01:40:56.165396 29109 process.cpp:3570] Handling HTTP event for process
>> 'master' with path: '/master/api/v1/scheduler'
>> I1103 01:40:56.167019 29109 http.cpp:391] HTTP POST for
>> /master/api/v1/scheduler from 172.17.0.2:45772
>> I1103 01:40:56.167212 29109 master.cpp:4867] Processing ACKNOWLEDGE call
>> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) on agent
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
>> I1103 01:40:56.168145 29110 status_update_manager.cpp:395] Received
>> status update acknowledgement (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
>> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> I1103 01:40:56.168666 29110 slave.cpp:3022] Status update manager
>> successfully handled status update acknowledgement (UUID:
>> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
>> 846a812b-b562-4782-b0fd-9e760a52306e of framework
>> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
>> @ 0x2b568528ffbd google::LogMessage::Flush()
>> @ 0x2b5685293438 google::LogMessageFatal::~LogMessageFatal()
>> @ 0xcb1fc4 google::CheckNotNull<>()
>> @ 0xc93ba6 process::Owned<>::operator->()
>> @ 0xc9100c mesos::internal::tests::TestCo
>> ntainerizer::_launch()
>> @ 0xc98182 _ZN7testing8internal12InvokeHe
>> lperIN7process6FutureIbEESt5tupleIJRKN5mesos11ContainerIDERK
>> 6OptionINS6_8TaskInfoEERKNS6_12ExecutorInfoERKSsRKSA_ISsERKN
>> S6_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISI_SsEEEbEEE1
>> 2InvokeMethodINS6_8internal5tests17TestContainerizerEMS14_
>> FS4_S9_SE_SH_SJ_SM_SP_SY_bEEES4_PT_T0_RKSZ_
>> @ 0xc98014 _ZNK7testing8internal18InvokeM
>> ethodActionIN5mesos8internal5tests17TestContainerizerEMS5_FN
>> 7process6FutureIbEERKNS2_11ContainerIDERK6OptionINS2_8TaskIn
>> foEERKNS2_12ExecutorInfoERKSsRKSC_ISsERKNS2_7SlaveIDERKSt3ma
>> pISsSsSt4lessISsESaISt4pairISK_SsEEEbEE7PerformIS8_St5tupleI
>> JSB_SG_SJ_SL_SO_SR_S10_bEEEET_RKT0_
>> @ 0xc97fb6 _ZN7testing17PolymorphicAction
>> INS_8internal18InvokeMethodActionIN5mesos8internal5tests17Te
>> stContainerizerEMS6_FN7process6FutureIbEERKNS3_11ContainerID
>> ERK6OptionINS3_8TaskInfoEERKNS3_12ExecutorInfoERKSsRKSD_ISsE
>> RKNS3_7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISL_SsEEEbE
>> EEE15MonomorphicImplIFS9_SC_SH_SK_SM_SP_SS_S11_bEE7Perform
>> ERKSt5tupleIJSC_SH_SK_SM_SP_SS_S11_bEE
>> @ 0xb51d58 _ZNK7testing6ActionIFN7process
>> 6FutureIbEERKN5mesos11ContainerIDERK6OptionINS4_8TaskInfoEER
>> KNS4_12ExecutorInfoERKSsRKS8_ISsERKNS4_7SlaveIDERKSt3mapISsS
>> sSt4lessISsESaISt4pairISG_SsEEEbEE7PerformERKSt5tupleIJS7_
>> SC_SF_SH_SK_SN_SW_bEE
>> @ 0xb51b6a testing::internal::ActionResul
>> tHolder<>::PerformAction<>()
>> @ 0xb4ca1c testing::internal::FunctionMoc
>> kerBase<>::UntypedPerformAction()
>> @ 0x1daa619 testing::internal::UntypedFunc
>> tionMockerBase::UntypedInvokeWith()
>> @ 0xb4995b _ZN7testing8internal18Function
>> MockerBaseIFN7process6FutureIbEERKN5mesos11ContainerIDERK6Op
>> tionINS5_8TaskInfoEERKNS5_12ExecutorInfoERKSsRKS9_ISsERKNS5_
>> 7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISH_SsEEEbEE10Inv
>> okeWithERKSt5tupleIJS8_SD_SG_SI_SL_SO_SX_bEE
>> @ 0xb49916 testing::internal::FunctionMocker<>::Invoke()
>> @ 0xc9712f mesos::internal::tests::TestCo
>> ntainerizer::launch()
>> @ 0x2b5684298b9b mesos::internal::slave::Framew
>> ork::launchExecutor()
>> @ 0x2b568429607e mesos::internal::slave::Slave::_run()
>> @ 0x2b56843265a0 _ZZN7process8dispatchIN5mesos8
>> internal5slave5SlaveERKNS_6FutureIbEERKNS1_13FrameworkInfoER
>> KNS1_12ExecutorInfoERK6OptionINS1_8TaskInfoEERKSF_INS1_13Tas
>> kGroupInfoEES6_S9_SC_SH_SL_EEvRKNS_3PIDIT_EEMSP_FvT0_T1_
>> T2_T3_T4_ET5_T6_T7_T8_T9_ENKUlPNS_11ProcessBaseEE_clES16_
>> @ 0x2b5684326062 _ZNSt17_Function_handlerIFvPN7
>> process11ProcessBaseEEZNS0_8dispatchIN5mesos8internal5slave5
>> SlaveERKNS0_6FutureIbEERKNS5_13FrameworkInfoERKNS5_12Executo
>> rInfoERK6OptionINS5_8TaskInfoEERKSJ_INS5_13TaskGroupInfoEESA
>> _SD_SG_SL_SP_EEvRKNS0_3PIDIT_EEMST_FvT0_T1_T2_T3_T4_ET5_T6_
>> T7_T8_T9_EUlS2_E_E9_M_invokeERKSt9_Any_dataS2_
>> @ 0x2b56851b2dd8 std::function<>::operator()()
>> @ 0x2b5685199f74 process::ProcessBase::visit()
>> @ 0x2b56851f84ee process::DispatchEvent::visit()
>> @ 0x8dabe1 process::ProcessBase::serve()
>> @ 0x2b5685197c64 process::ProcessManager::resume()
>> @ 0x2b56851a28cc process::ProcessManager::init_
>> threads()::$_0::operator()()
>> @ 0x2b56851a27d5 _ZNSt12_Bind_simpleIFZN7proces
>> s14ProcessManager12init_threadsEvE3$_0vEE9_M_
>> invokeIJEEEvSt12_Index_tupleIJXspT_EEE
>> @ 0x2b56851a27a5 std::_Bind_simple<>::operator()()
>> @ 0x2b56851a277c std::thread::_Impl<>::_M_run()
>> @ 0x2b5686d10a60 (unknown)
>> @ 0x2b5687487184 start_thread
>> make[4]: *** [check-local] Aborted
>> make[4]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
>> make[3]: *** [check-am] Error 2
>> make[3]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
>> make[2]: *** [check] Error 2
>> make[2]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
>> make[1]: *** [check-recursive] Error 1
>> make[1]: Leaving directory `/mesos/mesos-1.2.0/_build'
>> make: *** [distcheck] Error 1
>> + docker rmi mesos-1478134041-23345
>> Untagged: mesos-1478134041-23345:latest
>> Deleted: sha256:74596d8e8b76315d1115980623d7e5af9150c10d54203c726895a
>> a5cac6da79e
>> Deleted: sha256:7a0b0a4cf4ac2c9897b51052cb68965f0b4e8a305ff6db16b7d53
>> 0709b6e406c
>> Deleted: sha256:567f543714141c74573998d4209337287ffbe0e8ccd9ee8e4b044
>> cc6e872c0e5
>> Deleted: sha256:fbd8be03b0d3f1c70dea858388d951374325b608d3e56c6b51fd1
>> 02b193afed2
>> Deleted: sha256:7c7aee38843610c581d675c5620d8981f65d7fda18fc0fa05a264
>> fb510dd927c
>> Deleted: sha256:654d69e9689193c66be28a0fff218d21a5a119fbeecdc26a8a4cf
>> 34b3c2cd104
>> Deleted: sha256:4f5e0007d3903ea948c9fa13032fad2b1244dc0d4dbbee0673cd3
>> 470c8f3c24a
>> Deleted: sha256:38239a48898a69f2bf752d732a101a22038fc2d8e023b6c789987
>> daa2332ee1b
>> Deleted: sha256:329c7b62b50ac8578b3499838a1f12435013f0b4f429da2efac3d
>> 76673800950
>> Deleted: sha256:45dc8a92ffad997c6e48598af8e4665ff06cbb6d6316351de7103
>> 17f420367a2
>> Deleted: sha256:bc4e9bad738f327a22d9bb1204cbb3787b6b04fcda850497d7a6f
>> d54b1d9b9f3
>> Deleted: sha256:708772f854104da0e8b60eb0a9fb12c5cd85fca258ed74289e0ca
>> dcab12bb2f9
>> Deleted: sha256:5498b1ca089ca60b79713f6a57bc63036a89079ae1afff2ec21d4
>> 42d5fa531f1
>> Deleted: sha256:7830805a402ac02a943fc9dbac0ff1dd46c4cfd67af2fb4e36b9d
>> 2e43dd7a4d5
>> Deleted: sha256:354a09c0839db0533726726c863f23138aad1656c38ee8f3c16b0
>> 4b1766a0109
>> Deleted: sha256:73999f055e575a9767df52d22b67980d2b536b8ac1eeda0e9af1e
>> a623fc939b9
>> Deleted: sha256:c915f730bb38299ca517a3f295fcdd21e0078d959381ae5a9aff0
>> f39b4748bd5
>> Deleted: sha256:ca688ce3b2900d35c52872f38fd4af8530b780bb362e6e1a42687
>> d4f5e4082e8
>> Deleted: sha256:d2c33d1bb8213d5ed68c888aa09a65370a36d6ab4311704d2f106
>> e7ef94cb6fc
>> Deleted: sha256:67e4c09745aa1fd979f24bfae0812811bd44377dafd43130c0c27
>> 69e820d252a
>> Deleted: sha256:958624053829cf5b33921f84ad226d1835a271befa6856f591fc0
>> b4e1981c349
>> Deleted: sha256:a59579b1b2fd24a0570169c6a97d2ba86c88f44a9ac159c792f37
>> 71c9d3585a7
>> Deleted: sha256:4edbd39dc9e6dcb72d14b8e207b6705bace589ce1dc8cfe4525ff
>> ac3d216f471
>> Deleted: sha256:92fc49f6ea70503f089441fdcdfdcf8ebc3875060deefaf5f355c
>> 4a9b45314e8
>> Build step 'Execute shell' marked build as failure
>>
>
>
Re: Build failed in Jenkins: Mesos » autotools,clang,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,ubuntu:14.04,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2873
Posted by Benjamin Mahler <bm...@apache.org>.
This test uses two mock executors across two agents, which means the agents
may concurrently call into the TestContainerizer to launch the container.
The TestContainerizer is not a Process and so we may concurrently modify
its data structures, I suspect that the following executes concurrently and
leads to the crash:
https://github.com/apache/mesos/blob/44242c058158727ce013bd51764368f5e120ee75/src/tests/containerizer.cpp#L131
On Wed, Nov 2, 2016 at 6:41 PM, Apache Jenkins Server <
jenkins@builds.apache.org> wrote:
> See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,
> COMPILER=clang,CONFIGURATION=--verbose%20--enable-libevent%
> 20--enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=
> ubuntu%3A14.04,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-
> us1)&&(!ubuntu-6)/2873/changes>
>
> Changes:
>
> [josephwu] Fixed some markdown whitespace style.
>
> [josephwu] Fixed a typo in `gpu-support.md`.
>
> [josephwu] Fixed FD leak due to exiting early in Mesos fetcher.
>
> ------------------------------------------
> [...truncated 78174 lines...]
> I1103 01:40:55.530350 29098 slave.cpp:974] Authenticating with master
> master@172.17.0.2:58302
> I1103 01:40:55.530432 29098 slave.cpp:985] Using default CRAM-MD5
> authenticatee
> I1103 01:40:55.530627 29098 slave.cpp:947] Detecting new master
> I1103 01:40:55.530675 29108 authenticatee.cpp:121] Creating new client
> SASL connection
> I1103 01:40:55.530743 29098 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> I1103 01:40:55.530961 29099 master.cpp:6742] Authenticating slave(150)@
> 172.17.0.2:58302
> I1103 01:40:55.531070 29112 authenticator.cpp:414] Starting authentication
> session for crammd5-authenticatee(357)@172.17.0.2:58302
> I1103 01:40:55.531328 29106 authenticator.cpp:98] Creating new server SASL
> connection
> I1103 01:40:55.531561 29108 authenticatee.cpp:213] Received SASL
> authentication mechanisms: CRAM-MD5
> I1103 01:40:55.531604 29108 authenticatee.cpp:239] Attempting to
> authenticate with mechanism 'CRAM-MD5'
> I1103 01:40:55.531713 29101 authenticator.cpp:204] Received SASL
> authentication start
> I1103 01:40:55.531805 29101 authenticator.cpp:326] Authentication requires
> more steps
> I1103 01:40:55.531921 29108 authenticatee.cpp:259] Received SASL
> authentication step
> I1103 01:40:55.532120 29101 authenticator.cpp:232] Received SASL
> authentication step
> I1103 01:40:55.532155 29101 auxprop.cpp:109] Request to lookup properties
> for user: 'test-principal' realm: '3a1c598ce334' server FQDN:
> '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: false
> I1103 01:40:55.532179 29101 auxprop.cpp:181] Looking up auxiliary property
> '*userPassword'
> I1103 01:40:55.532233 29101 auxprop.cpp:181] Looking up auxiliary property
> '*cmusaslsecretCRAM-MD5'
> I1103 01:40:55.532266 29101 auxprop.cpp:109] Request to lookup properties
> for user: 'test-principal' realm: '3a1c598ce334' server FQDN:
> '3a1c598ce334' SASL_AUXPROP_VERIFY_AGAINST_HASH: false
> SASL_AUXPROP_OVERRIDE: false SASL_AUXPROP_AUTHZID: true
> I1103 01:40:55.532289 29101 auxprop.cpp:131] Skipping auxiliary property
> '*userPassword' since SASL_AUXPROP_AUTHZID == true
> I1103 01:40:55.532305 29101 auxprop.cpp:131] Skipping auxiliary property
> '*cmusaslsecretCRAM-MD5' since SASL_AUXPROP_AUTHZID == true
> I1103 01:40:55.532335 29101 authenticator.cpp:318] Authentication success
> I1103 01:40:55.532413 29110 authenticatee.cpp:299] Authentication success
> I1103 01:40:55.532467 29108 master.cpp:6772] Successfully authenticated
> principal 'test-principal' at slave(150)@172.17.0.2:58302
> I1103 01:40:55.532536 29111 authenticator.cpp:432] Authentication session
> cleanup for crammd5-authenticatee(357)@172.17.0.2:58302
> I1103 01:40:55.532755 29098 slave.cpp:1069] Successfully authenticated
> with master master@172.17.0.2:58302
> I1103 01:40:55.532997 29098 slave.cpp:1483] Will retry registration in
> 12.590371ms if necessary
> I1103 01:40:55.533179 29108 master.cpp:5151] Registering agent at
> slave(150)@172.17.0.2:58302 (maintenance-host-2) with id
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
> I1103 01:40:55.533572 29112 registrar.cpp:461] Applied 1 operations in
> 94467ns; attempting to update the registry
> I1103 01:40:55.546341 29107 slave.cpp:1483] Will retry registration in
> 36.501523ms if necessary
> I1103 01:40:55.546461 29099 master.cpp:5139] Ignoring register agent
> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
> admission is already in progress
> I1103 01:40:55.565403 29097 leveldb.cpp:341] Persisting action (16 bytes)
> to leveldb took 48.099208ms
> I1103 01:40:55.565495 29097 replica.cpp:708] Persisted action TRUNCATE at
> position 4
> I1103 01:40:55.566788 29097 replica.cpp:691] Replica received learned
> notice for position 4 from @0.0.0.0:0
> I1103 01:40:55.583937 29101 slave.cpp:1483] Will retry registration in
> 26.127711ms if necessary
> I1103 01:40:55.584123 29112 master.cpp:5139] Ignoring register agent
> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
> admission is already in progress
> I1103 01:40:55.609695 29097 leveldb.cpp:341] Persisting action (18 bytes)
> to leveldb took 42.905697ms
> I1103 01:40:55.609860 29097 leveldb.cpp:399] Deleting ~2 keys from leveldb
> took 96623ns
> I1103 01:40:55.609899 29097 replica.cpp:708] Persisted action TRUNCATE at
> position 4
> I1103 01:40:55.611063 29106 log.cpp:577] Attempting to append 513 bytes to
> the log
> I1103 01:40:55.611229 29097 coordinator.cpp:348] Coordinator attempting to
> write APPEND action at position 5
> I1103 01:40:55.611498 29100 slave.cpp:1483] Will retry registration in
> 85.55417ms if necessary
> I1103 01:40:55.612069 29105 master.cpp:5139] Ignoring register agent
> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
> admission is already in progress
> I1103 01:40:55.612313 29100 replica.cpp:537] Replica received write
> request for position 5 from __req_res__(2518)@172.17.0.2:58302
> I1103 01:40:55.657845 29100 leveldb.cpp:341] Persisting action (532 bytes)
> to leveldb took 45.517822ms
> I1103 01:40:55.657938 29100 replica.cpp:708] Persisted action APPEND at
> position 5
> I1103 01:40:55.658681 29103 replica.cpp:691] Replica received learned
> notice for position 5 from @0.0.0.0:0
> I1103 01:40:55.698050 29112 slave.cpp:1483] Will retry registration in
> 100.32384ms if necessary
> I1103 01:40:55.698319 29101 master.cpp:5139] Ignoring register agent
> message from slave(150)@172.17.0.2:58302 (maintenance-host-2) as
> admission is already in progress
> I1103 01:40:55.705086 29103 leveldb.cpp:341] Persisting action (534 bytes)
> to leveldb took 46.384557ms
> I1103 01:40:55.705157 29103 replica.cpp:708] Persisted action APPEND at
> position 5
> I1103 01:40:55.707480 29098 registrar.cpp:506] Successfully updated the
> registry in 173824us
> I1103 01:40:55.707741 29100 log.cpp:596] Attempting to truncate the log to
> 5
> I1103 01:40:55.708029 29110 coordinator.cpp:348] Coordinator attempting to
> write TRUNCATE action at position 6
> I1103 01:40:55.708501 29097 slave.cpp:4251] Received ping from
> slave-observer(151)@172.17.0.2:58302
> I1103 01:40:55.708528 29112 master.cpp:5222] Registered agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
> (maintenance-host-2) with cpus(*):2; mem(*):1024; disk(*):1024;
> ports(*):[31000-32000]
> I1103 01:40:55.708796 29097 slave.cpp:1115] Registered with master
> master@172.17.0.2:58302; given agent ID 3167a687-904b-4b57-bc0f-
> 91b67dc7e41d-S1
> I1103 01:40:55.708822 29097 fetcher.cpp:86] Clearing fetcher cache
> I1103 01:40:55.708889 29103 replica.cpp:537] Replica received write
> request for position 6 from __req_res__(2519)@172.17.0.2:58302
> I1103 01:40:55.708935 29100 hierarchical.cpp:485] Added agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) with
> cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000] (allocated: {})
> I1103 01:40:55.709065 29105 status_update_manager.cpp:184] Resuming
> sending status updates
> I1103 01:40:55.709139 29100 hierarchical.cpp:1694] No allocations performed
> I1103 01:40:55.709204 29100 hierarchical.cpp:1309] Performed allocation
> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 229932ns
> I1103 01:40:55.709316 29097 slave.cpp:1138] Checkpointing SlaveInfo to
> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/
> meta/slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/slave.info'
> I1103 01:40:55.709650 29097 slave.cpp:1175] Forwarding total
> oversubscribed resources {}
> I1103 01:40:55.709787 29097 master.cpp:5621] Received update of agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
> (maintenance-host-2) with total oversubscribed resources {}
> I1103 01:40:55.710000 29108 hierarchical.cpp:555] Agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2) updated with
> oversubscribed resources {} (total: cpus(*):2; mem(*):1024; disk(*):1024;
> ports(*):[31000-32000], allocated: {})
> I1103 01:40:55.710144 29108 hierarchical.cpp:1694] No allocations performed
> I1103 01:40:55.710199 29108 hierarchical.cpp:1309] Performed allocation
> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 157491ns
> I1103 01:40:55.711632 29105 process.cpp:3570] Handling HTTP event for
> process 'master' with path: '/master/maintenance/schedule'
> I1103 01:40:55.712441 29111 http.cpp:391] HTTP POST for
> /master/maintenance/schedule from 172.17.0.2:45755
> I1103 01:40:55.713325 29110 registrar.cpp:461] Applied 1 operations in
> 165622ns; attempting to update the registry
> I1103 01:40:55.756808 29103 leveldb.cpp:341] Persisting action (16 bytes)
> to leveldb took 47.899715ms
> I1103 01:40:55.756888 29103 replica.cpp:708] Persisted action TRUNCATE at
> position 6
> I1103 01:40:55.758083 29103 replica.cpp:691] Replica received learned
> notice for position 6 from @0.0.0.0:0
> I1103 01:40:55.807322 29103 leveldb.cpp:341] Persisting action (18 bytes)
> to leveldb took 49.23844ms
> I1103 01:40:55.807481 29103 leveldb.cpp:399] Deleting ~2 keys from leveldb
> took 90240ns
> I1103 01:40:55.807517 29103 replica.cpp:708] Persisted action TRUNCATE at
> position 6
> I1103 01:40:55.808619 29101 log.cpp:577] Attempting to append 732 bytes to
> the log
> I1103 01:40:55.808737 29097 coordinator.cpp:348] Coordinator attempting to
> write APPEND action at position 7
> I1103 01:40:55.809571 29109 replica.cpp:537] Replica received write
> request for position 7 from __req_res__(2520)@172.17.0.2:58302
> I1103 01:40:55.810784 29098 hierarchical.cpp:1694] No allocations performed
> I1103 01:40:55.810860 29098 hierarchical.cpp:1286] Performed allocation
> for 2 agents in 282405ns
> I1103 01:40:55.870082 29109 leveldb.cpp:341] Persisting action (751 bytes)
> to leveldb took 60.489282ms
> I1103 01:40:55.870177 29109 replica.cpp:708] Persisted action APPEND at
> position 7
> I1103 01:40:55.871421 29109 replica.cpp:691] Replica received learned
> notice for position 7 from @0.0.0.0:0
> I1103 01:40:55.976619 29109 leveldb.cpp:341] Persisting action (753 bytes)
> to leveldb took 105.201942ms
> I1103 01:40:55.976718 29109 replica.cpp:708] Persisted action APPEND at
> position 7
> I1103 01:40:55.979708 29109 registrar.cpp:506] Successfully updated the
> registry in 266.301952ms
> I1103 01:40:55.979878 29103 log.cpp:596] Attempting to truncate the log to
> 7
> I1103 01:40:55.979991 29106 coordinator.cpp:348] Coordinator attempting to
> write TRUNCATE action at position 8
> I1103 01:40:55.980255 29112 master.cpp:5683] Updating unavailability of
> agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@
> 172.17.0.2:58302 (maintenance-host), starting at 2449.0107815972weeks
> I1103 01:40:55.980481 29112 master.cpp:5683] Updating unavailability of
> agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@
> 172.17.0.2:58302 (maintenance-host-2), starting at 2449.0107815972weeks
> I1103 01:40:55.980818 29097 replica.cpp:537] Replica received write
> request for position 8 from __req_res__(2521)@172.17.0.2:58302
> I1103 01:40:55.981092 29103 hierarchical.cpp:1694] No allocations performed
> I1103 01:40:55.981150 29103 hierarchical.cpp:1309] Performed allocation
> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 in 199516ns
> I1103 01:40:55.981317 29103 hierarchical.cpp:1694] No allocations performed
> I1103 01:40:55.981367 29103 hierarchical.cpp:1309] Performed allocation
> for agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 in 144365ns
> I1103 01:40:55.982864 29079 scheduler.cpp:176] Version: 1.2.0
> I1103 01:40:55.983412 29112 scheduler.cpp:469] New master detected at
> master@172.17.0.2:58302
> I1103 01:40:55.983438 29112 scheduler.cpp:478] Waiting for 0ns before
> initiating a re-(connection) attempt with the master
> I1103 01:40:55.984688 29098 scheduler.cpp:353] Connected with the master
> at http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:55.985872 29109 scheduler.cpp:235] Sending SUBSCRIBE call to
> http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:55.986587 29099 process.cpp:3570] Handling HTTP event for
> process 'master' with path: '/master/api/v1/scheduler'
> I1103 01:40:55.987659 29108 http.cpp:391] HTTP POST for
> /master/api/v1/scheduler from 172.17.0.2:45771
> I1103 01:40:55.988019 29108 master.cpp:2329] Received subscription request
> for HTTP framework 'default'
> I1103 01:40:55.988088 29108 master.cpp:2069] Authorizing framework
> principal 'test-principal' to receive offers for role '*'
> I1103 01:40:55.988632 29099 master.cpp:2427] Subscribing framework
> 'default' with checkpointing disabled and capabilities [ ]
> I1103 01:40:55.989305 29105 hierarchical.cpp:275] Added framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:55.989459 29099 master.hpp:2161] Sending heartbeat to
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:55.990151 29108 scheduler.cpp:675] Enqueuing event SUBSCRIBED
> received from http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:55.990648 29108 scheduler.cpp:675] Enqueuing event HEARTBEAT
> received from http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:55.991364 29105 hierarchical.cpp:1286] Performed allocation
> for 2 agents in 2.044414ms
> I1103 01:40:55.992657 29100 master.cpp:6571] Sending 2 offers to framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
> I1103 01:40:55.994067 29100 master.cpp:6661] Sending 2 inverse offers to
> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default)
> I1103 01:40:55.995620 29101 scheduler.cpp:675] Enqueuing event OFFERS
> received from http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:55.996536 29101 scheduler.cpp:675] Enqueuing event
> INVERSE_OFFERS received from http://172.17.0.2:58302/
> master/api/v1/scheduler
> I1103 01:40:56.060276 29097 leveldb.cpp:341] Persisting action (16 bytes)
> to leveldb took 79.452553ms
> I1103 01:40:56.060681 29097 replica.cpp:708] Persisted action TRUNCATE at
> position 8
> I1103 01:40:56.061957 29102 replica.cpp:691] Replica received learned
> notice for position 8 from @0.0.0.0:0
> I1103 01:40:56.118927 29102 leveldb.cpp:341] Persisting action (18 bytes)
> to leveldb took 56.925572ms
> I1103 01:40:56.119022 29102 leveldb.cpp:399] Deleting ~2 keys from leveldb
> took 51598ns
> I1103 01:40:56.119046 29102 replica.cpp:708] Persisted action TRUNCATE at
> position 8
> I1103 01:40:56.120704 29099 hierarchical.cpp:1694] No allocations performed
> I1103 01:40:56.121099 29099 hierarchical.cpp:1286] Performed allocation
> for 2 agents in 610047ns
> I1103 01:40:56.122525 29097 scheduler.cpp:235] Sending ACCEPT call to
> http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:56.123138 29097 scheduler.cpp:235] Sending ACCEPT call to
> http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:56.124460 29098 process.cpp:3570] Handling HTTP event for
> process 'master' with path: '/master/api/v1/scheduler'
> I1103 01:40:56.124822 29098 process.cpp:3570] Handling HTTP event for
> process 'master' with path: '/master/api/v1/scheduler'
> I1103 01:40:56.126590 29098 http.cpp:391] HTTP POST for
> /master/api/v1/scheduler from 172.17.0.2:45772
> I1103 01:40:56.127682 29098 master.cpp:3581] Processing ACCEPT call for
> offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O0 ] on agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 at slave(149)@172.17.0.2:58302
> (maintenance-host) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> (default)
> I1103 01:40:56.127856 29098 master.cpp:3173] Authorizing framework
> principal 'test-principal' to launch task 19b41cf9-ba0f-472e-8cae-
> 5ea70574b1b4
> I1103 01:40:56.128916 29098 http.cpp:391] HTTP POST for
> /master/api/v1/scheduler from 172.17.0.2:45772
> I1103 01:40:56.129622 29098 master.cpp:3581] Processing ACCEPT call for
> offers: [ 3167a687-904b-4b57-bc0f-91b67dc7e41d-O1 ] on agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@172.17.0.2:58302
> (maintenance-host-2) for framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> (default)
> I1103 01:40:56.129730 29098 master.cpp:3173] Authorizing framework
> principal 'test-principal' to launch task 846a812b-b562-4782-b0fd-
> 9e760a52306e
> W1103 01:40:56.131443 29098 validation.cpp:920] Executor 'executor-1' for
> task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less CPUs (None) than
> the minimum required (0.01). Please update your executor, as this will be
> mandatory in future releases.
> W1103 01:40:56.131489 29098 validation.cpp:932] Executor 'executor-1' for
> task '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' uses less memory (None) than
> the minimum required (32MB). Please update your executor, as this will be
> mandatory in future releases.
> I1103 01:40:56.132071 29098 master.cpp:8334] Adding task
> 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 with resources cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0 (maintenance-host)
> I1103 01:40:56.132442 29098 master.cpp:4230] Launching task
> 19b41cf9-ba0f-472e-8cae-5ea70574b1b4 of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> (default) with resources cpus(*):2; mem(*):1024; disk(*):1024;
> ports(*):[31000-32000] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S0
> at slave(149)@172.17.0.2:58302 (maintenance-host)
> W1103 01:40:56.133824 29098 validation.cpp:920] Executor 'executor-2' for
> task '846a812b-b562-4782-b0fd-9e760a52306e' uses less CPUs (None) than
> the minimum required (0.01). Please update your executor, as this will be
> mandatory in future releases.
> W1103 01:40:56.133858 29098 validation.cpp:932] Executor 'executor-2' for
> task '846a812b-b562-4782-b0fd-9e760a52306e' uses less memory (None) than
> the minimum required (32MB). Please update your executor, as this will be
> mandatory in future releases.
> I1103 01:40:56.134299 29097 slave.cpp:1547] Got assigned task
> '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.134332 29098 master.cpp:8334] Adding task
> 846a812b-b562-4782-b0fd-9e760a52306e with resources cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000] on agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 (maintenance-host-2)
> I1103 01:40:56.134572 29098 master.cpp:4230] Launching task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> (default) with resources cpus(*):2; mem(*):1024; disk(*):1024;
> ports(*):[31000-32000] on agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
> at slave(150)@172.17.0.2:58302 (maintenance-host-2)
> I1103 01:40:56.135084 29097 slave.cpp:1709] Launching task
> '19b41cf9-ba0f-472e-8cae-5ea70574b1b4' for framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.135145 29098 slave.cpp:1547] Got assigned task
> '846a812b-b562-4782-b0fd-9e760a52306e' for framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.135660 29098 slave.cpp:1709] Launching task
> '846a812b-b562-4782-b0fd-9e760a52306e' for framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.135871 29097 paths.cpp:536] Trying to chown
> '/tmp/MasterMaintenanceTest_InverseOffersFilters_B4KX1H/
> slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S0/
> frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/
> executors/executor-1/runs/fbb81e53-b134-4196-9385-791612794545' to user
> 'mesos'
> I1103 01:40:56.138550 29098 paths.cpp:536] Trying to chown
> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/
> slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/
> frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/
> executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4' to user
> 'mesos'
> I1103 01:40:56.144282 29098 slave.cpp:6307] Launching executor
> 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 with
> resources {} in work directory '/tmp/MasterMaintenanceTest_
> InverseOffersFilters_xHOK8S/slaves/3167a687-904b-4b57-
> bc0f-91b67dc7e41d-S1/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/
> executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
> I1103 01:40:56.144282 29097 slave.cpp:6307] Launching executor
> 'executor-1' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 with
> resources {} in work directory '/tmp/MasterMaintenanceTest_
> InverseOffersFilters_B4KX1H/slaves/3167a687-904b-4b57-
> bc0f-91b67dc7e41d-S0/frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/
> executors/executor-1/runs/fbb81e53-b134-4196-9385-791612794545'
> F1103 01:40:56.144891 29097 owned.hpp:110] Check failed: 'get()' Must be
> non NULL
> *** Check failure stack trace: ***
> I1103 01:40:56.147233 29098 exec.cpp:162] Version: 1.2.0
> I1103 01:40:56.147598 29112 exec.cpp:212] Executor started at:
> executor(60)@172.17.0.2:58302 with pid 29079
> I1103 01:40:56.147956 29098 slave.cpp:2031] Queued task
> '846a812b-b562-4782-b0fd-9e760a52306e' for executor 'executor-2' of
> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.148043 29098 slave.cpp:868] Successfully attached file
> '/tmp/MasterMaintenanceTest_InverseOffersFilters_xHOK8S/
> slaves/3167a687-904b-4b57-bc0f-91b67dc7e41d-S1/
> frameworks/3167a687-904b-4b57-bc0f-91b67dc7e41d-0000/
> executors/executor-2/runs/c7d8cc2e-b173-4dbe-b5ae-e45402c1cdf4'
> I1103 01:40:56.148277 29098 slave.cpp:3305] Got registration for executor
> 'executor-2' of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 from
> executor(60)@172.17.0.2:58302
> I1103 01:40:56.148833 29105 exec.cpp:237] Executor registered on agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
> I1103 01:40:56.148895 29105 exec.cpp:249] Executor::registered took 29894ns
> I1103 01:40:56.149114 29098 slave.cpp:2247] Sending queued task
> '846a812b-b562-4782-b0fd-9e760a52306e' to executor 'executor-2' of
> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 at executor(60)@
> 172.17.0.2:58302
> I1103 01:40:56.149495 29106 exec.cpp:324] Executor asked to run task
> '846a812b-b562-4782-b0fd-9e760a52306e'
> I1103 01:40:56.149608 29106 exec.cpp:333] Executor::launchTask took 98162ns
> I1103 01:40:56.149705 29106 exec.cpp:550] Executor sending status update
> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-
> 91b67dc7e41d-0000
> I1103 01:40:56.149987 29106 slave.cpp:3740] Handling status update
> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> from executor(60)@172.17.0.2:58302
> I1103 01:40:56.150545 29098 status_update_manager.cpp:323] Received
> status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.150606 29098 status_update_manager.cpp:500] Creating
> StatusUpdate stream for task 846a812b-b562-4782-b0fd-9e760a52306e of
> framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.151116 29098 status_update_manager.cpp:377] Forwarding
> update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> to the agent
> @ 0x2b56852902fd google::LogMessage::Fail()
> I1103 01:40:56.151470 29101 slave.cpp:4169] Forwarding the update
> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> to master@172.17.0.2:58302
> I1103 01:40:56.151757 29101 slave.cpp:4063] Status update manager
> successfully handled status update TASK_RUNNING (UUID:
> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e
> of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.151860 29101 slave.cpp:4079] Sending acknowledgement for
> status update TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26)
> for task 846a812b-b562-4782-b0fd-9e760a52306e of framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 to executor(60)@172.17.0.2:58302
> I1103 01:40:56.151954 29111 master.cpp:5757] Status update TASK_RUNNING
> (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> from agent 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1 at slave(150)@
> 172.17.0.2:58302 (maintenance-host-2)
> I1103 01:40:56.152041 29111 master.cpp:5819] Forwarding status update
> TASK_RUNNING (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-
> 91b67dc7e41d-0000
> I1103 01:40:56.152123 29100 exec.cpp:373] Executor received status update
> acknowledgement 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-
> 91b67dc7e41d-0000
> I1103 01:40:56.152546 29111 master.cpp:7712] Updating the state of task
> 846a812b-b562-4782-b0fd-9e760a52306e of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
> I1103 01:40:56.153616 29103 scheduler.cpp:675] Enqueuing event UPDATE
> received from http://172.17.0.2:58302/master/api/v1/scheduler
> @ 0x2b568528f6de google::LogMessage::SendToLog()
> I1103 01:40:56.164335 29109 scheduler.cpp:235] Sending ACKNOWLEDGE call to
> http://172.17.0.2:58302/master/api/v1/scheduler
> I1103 01:40:56.165396 29109 process.cpp:3570] Handling HTTP event for
> process 'master' with path: '/master/api/v1/scheduler'
> I1103 01:40:56.167019 29109 http.cpp:391] HTTP POST for
> /master/api/v1/scheduler from 172.17.0.2:45772
> I1103 01:40:56.167212 29109 master.cpp:4867] Processing ACKNOWLEDGE call
> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26 for task 846a812b-b562-4782-b0fd-9e760a52306e
> of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000 (default) on agent
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-S1
> I1103 01:40:56.168145 29110 status_update_manager.cpp:395] Received status
> update acknowledgement (UUID: 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for
> task 846a812b-b562-4782-b0fd-9e760a52306e of framework
> 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> I1103 01:40:56.168666 29110 slave.cpp:3022] Status update manager
> successfully handled status update acknowledgement (UUID:
> 1fe05bba-4be1-4c4e-a847-e4e3289b2d26) for task 846a812b-b562-4782-b0fd-9e760a52306e
> of framework 3167a687-904b-4b57-bc0f-91b67dc7e41d-0000
> @ 0x2b568528ffbd google::LogMessage::Flush()
> @ 0x2b5685293438 google::LogMessageFatal::~LogMessageFatal()
> @ 0xcb1fc4 google::CheckNotNull<>()
> @ 0xc93ba6 process::Owned<>::operator->()
> @ 0xc9100c mesos::internal::tests::
> TestContainerizer::_launch()
> @ 0xc98182 _ZN7testing8internal12InvokeHel
> perIN7process6FutureIbEESt5tupleIJRKN5mesos11ContainerIDERK6
> OptionINS6_8TaskInfoEERKNS6_12ExecutorInfoERKSsRKSA_ISsERKNS6_
> 7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISI_
> SsEEEbEEE12InvokeMethodINS6_8internal5tests17TestContainer
> izerEMS14_FS4_S9_SE_SH_SJ_SM_SP_SY_bEEES4_PT_T0_RKSZ_
> @ 0xc98014 _ZNK7testing8internal18InvokeMe
> thodActionIN5mesos8internal5tests17TestContainerizerEMS5_
> FN7process6FutureIbEERKNS2_11ContainerIDERK6OptionINS2_8TaskInfoEERKNS2_
> 12ExecutorInfoERKSsRKSC_ISsERKNS2_7SlaveIDERKSt3mapISsSsSt4lessI
> SsESaISt4pairISK_SsEEEbEE7PerformIS8_St5tupleIJSB_SG_SJ_SL_SO_SR_
> S10_bEEEET_RKT0_
> @ 0xc97fb6 _ZN7testing17PolymorphicActionINS_
> 8internal18InvokeMethodActionIN5mesos8internal5tests17TestCo
> ntainerizerEMS6_FN7process6FutureIbEERKNS3_11ContainerIDERK6OptionINS3_
> 8TaskInfoEERKNS3_12ExecutorInfoERKSsRKSD_ISsERKNS3_
> 7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISL_
> SsEEEbEEEE15MonomorphicImplIFS9_SC_SH_SK_SM_SP_SS_S11_
> bEE7PerformERKSt5tupleIJSC_SH_SK_SM_SP_SS_S11_bEE
> @ 0xb51d58 _ZNK7testing6ActionIFN7process6
> FutureIbEERKN5mesos11ContainerIDERK6OptionINS4_8TaskInfoEERKNS4_
> 12ExecutorInfoERKSsRKS8_ISsERKNS4_7SlaveIDERKSt3mapISsSsSt4lessI
> SsESaISt4pairISG_SsEEEbEE7PerformERKSt5tupleIJS7_SC_SF_SH_SK_SN_SW_bEE
> @ 0xb51b6a testing::internal::ActionResultHolder<>::
> PerformAction<>()
> @ 0xb4ca1c testing::internal::FunctionMockerBase<>::
> UntypedPerformAction()
> @ 0x1daa619 testing::internal::UntypedFunctionMockerBase::
> UntypedInvokeWith()
> @ 0xb4995b _ZN7testing8internal18FunctionM
> ockerBaseIFN7process6FutureIbEERKN5mesos11ContainerIDERK6Opt
> ionINS5_8TaskInfoEERKNS5_12ExecutorInfoERKSsRKS9_ISsERKNS5_
> 7SlaveIDERKSt3mapISsSsSt4lessISsESaISt4pairISH_
> SsEEEbEE10InvokeWithERKSt5tupleIJS8_SD_SG_SI_SL_SO_SX_bEE
> @ 0xb49916 testing::internal::FunctionMocker<>::Invoke()
> @ 0xc9712f mesos::internal::tests::
> TestContainerizer::launch()
> @ 0x2b5684298b9b mesos::internal::slave::
> Framework::launchExecutor()
> @ 0x2b568429607e mesos::internal::slave::Slave::_run()
> @ 0x2b56843265a0 _ZZN7process8dispatchIN5mesos8i
> nternal5slave5SlaveERKNS_6FutureIbEERKNS1_13FrameworkInfoERKNS1_
> 12ExecutorInfoERK6OptionINS1_8TaskInfoEERKSF_INS1_
> 13TaskGroupInfoEES6_S9_SC_SH_SL_EEvRKNS_3PIDIT_EEMSP_FvT0_
> T1_T2_T3_T4_ET5_T6_T7_T8_T9_ENKUlPNS_11ProcessBaseEE_clES16_
> @ 0x2b5684326062 _ZNSt17_Function_handlerIFvPN7process11ProcessB
> aseEEZNS0_8dispatchIN5mesos8internal5slave5SlaveERKNS0_6FutureIbEERKNS5_
> 13FrameworkInfoERKNS5_12ExecutorInfoERK6OptionINS5_8TaskInfoEERKSJ_INS5_
> 13TaskGroupInfoEESA_SD_SG_SL_SP_EEvRKNS0_3PIDIT_EEMST_FvT0_
> T1_T2_T3_T4_ET5_T6_T7_T8_T9_EUlS2_E_E9_M_invokeERKSt9_Any_dataS2_
> @ 0x2b56851b2dd8 std::function<>::operator()()
> @ 0x2b5685199f74 process::ProcessBase::visit()
> @ 0x2b56851f84ee process::DispatchEvent::visit()
> @ 0x8dabe1 process::ProcessBase::serve()
> @ 0x2b5685197c64 process::ProcessManager::resume()
> @ 0x2b56851a28cc process::ProcessManager::init_
> threads()::$_0::operator()()
> @ 0x2b56851a27d5 _ZNSt12_Bind_simpleIFZN7process14ProcessMan
> ager12init_threadsEvE3$_0vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE
> @ 0x2b56851a27a5 std::_Bind_simple<>::operator()()
> @ 0x2b56851a277c std::thread::_Impl<>::_M_run()
> @ 0x2b5686d10a60 (unknown)
> @ 0x2b5687487184 start_thread
> make[4]: *** [check-local] Aborted
> make[4]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> make[3]: *** [check-am] Error 2
> make[3]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> make[2]: *** [check] Error 2
> make[2]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> make[1]: *** [check-recursive] Error 1
> make[1]: Leaving directory `/mesos/mesos-1.2.0/_build'
> make: *** [distcheck] Error 1
> + docker rmi mesos-1478134041-23345
> Untagged: mesos-1478134041-23345:latest
> Deleted: sha256:74596d8e8b76315d1115980623d7e5
> af9150c10d54203c726895aa5cac6da79e
> Deleted: sha256:7a0b0a4cf4ac2c9897b51052cb6896
> 5f0b4e8a305ff6db16b7d530709b6e406c
> Deleted: sha256:567f543714141c74573998d4209337
> 287ffbe0e8ccd9ee8e4b044cc6e872c0e5
> Deleted: sha256:fbd8be03b0d3f1c70dea858388d951
> 374325b608d3e56c6b51fd102b193afed2
> Deleted: sha256:7c7aee38843610c581d675c5620d89
> 81f65d7fda18fc0fa05a264fb510dd927c
> Deleted: sha256:654d69e9689193c66be28a0fff218d
> 21a5a119fbeecdc26a8a4cf34b3c2cd104
> Deleted: sha256:4f5e0007d3903ea948c9fa13032fad
> 2b1244dc0d4dbbee0673cd3470c8f3c24a
> Deleted: sha256:38239a48898a69f2bf752d732a101a
> 22038fc2d8e023b6c789987daa2332ee1b
> Deleted: sha256:329c7b62b50ac8578b3499838a1f12
> 435013f0b4f429da2efac3d76673800950
> Deleted: sha256:45dc8a92ffad997c6e48598af8e466
> 5ff06cbb6d6316351de710317f420367a2
> Deleted: sha256:bc4e9bad738f327a22d9bb1204cbb3
> 787b6b04fcda850497d7a6fd54b1d9b9f3
> Deleted: sha256:708772f854104da0e8b60eb0a9fb12
> c5cd85fca258ed74289e0cadcab12bb2f9
> Deleted: sha256:5498b1ca089ca60b79713f6a57bc63
> 036a89079ae1afff2ec21d442d5fa531f1
> Deleted: sha256:7830805a402ac02a943fc9dbac0ff1
> dd46c4cfd67af2fb4e36b9d2e43dd7a4d5
> Deleted: sha256:354a09c0839db0533726726c863f23
> 138aad1656c38ee8f3c16b04b1766a0109
> Deleted: sha256:73999f055e575a9767df52d22b6798
> 0d2b536b8ac1eeda0e9af1ea623fc939b9
> Deleted: sha256:c915f730bb38299ca517a3f295fcdd
> 21e0078d959381ae5a9aff0f39b4748bd5
> Deleted: sha256:ca688ce3b2900d35c52872f38fd4af
> 8530b780bb362e6e1a42687d4f5e4082e8
> Deleted: sha256:d2c33d1bb8213d5ed68c888aa09a65
> 370a36d6ab4311704d2f106e7ef94cb6fc
> Deleted: sha256:67e4c09745aa1fd979f24bfae08128
> 11bd44377dafd43130c0c2769e820d252a
> Deleted: sha256:958624053829cf5b33921f84ad226d
> 1835a271befa6856f591fc0b4e1981c349
> Deleted: sha256:a59579b1b2fd24a0570169c6a97d2b
> a86c88f44a9ac159c792f3771c9d3585a7
> Deleted: sha256:4edbd39dc9e6dcb72d14b8e207b670
> 5bace589ce1dc8cfe4525ffac3d216f471
> Deleted: sha256:92fc49f6ea70503f089441fdcdfdcf
> 8ebc3875060deefaf5f355c4a9b45314e8
> Build step 'Execute shell' marked build as failure
>