You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@mesos.apache.org by Douglas Nelson <it...@gmail.com> on 2016/07/18 22:13:22 UTC

Possible authentication bug

I have SSL enabled for mesos and for the most part everything seems to be
working fine. But when I stop a slave node for long enough that it shows up
with status LOST then I start up the slave again, registration with the
master fails:

I0718 15:51:45.646260 16791 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0718 15:51:45.646960 16791 authenticator.cpp:98] Creating new server SASL
connection
I0718 15:51:50.648329 16790 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0718 15:51:50.648696 16790 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded

It cycles through this over and over again until I restart the master node.
Is restarting the master the only way to handle re-authentication? I
expected it to be more automatic. Thanks!

Re: Possible authentication bug

Posted by Vinod Kone <vi...@apache.org>.

On Thu, Jul 21, 2016 at 4:49 PM, Douglas Nelson <it...@gmail.com> wrote:

> Just out of curiosity, is there a rough ETA for the stable release of
> 1.0.0? Or is anyone currently using rc2 in production?
>

I'm hoping to cut RC3 later today or tomorrow and barring any -ve votes do
the official release early next week.

Re: Possible authentication bug

Posted by Douglas Nelson <it...@gmail.com>.

I replaced the older mesos with 1.0.0-rc2 and it's working fine now. Thanks!

Just out of curiosity, is there a rough ETA for the stable release of
1.0.0? Or is anyone currently using rc2 in production?

On Thu, Jul 21, 2016 at 12:49 PM, Adam Bordelon <ad...@mesosphere.io> wrote:

> The flag is new in 1.0.0, so we'd have to backport it for 0.28 for you, or
> you'd have to try out 1.0 from master or an rc.
>
> On Thu, Jul 21, 2016 at 11:26 AM, Douglas Nelson <it...@gmail.com>
> wrote:
>
>> I tried setting --authentication_backoff_factor=9secs to experiment with
>> a different time. The mesos agent doesn't start at all when I set that
>> flag? I'm running mesos 0.28.2.
>>
>> On Wed, Jul 20, 2016 at 9:12 PM, Adam Bordelon <ad...@mesosphere.io>
>> wrote:
>>
>>> This does indeed look identical to the logs from previous occurrences of
>>> https://issues.apache.org/jira/browse/MESOS-2043
>>> This can happen when your master's message queue gets clogged (check
>>> /metrics/snapshot). The scheduler/agent begins authentication and starts
>>> its timeout counter. Eventually, the master gets the message out of its
>>> queue and asks the scheduler/agent for the next step in authentication. If
>>> more than 5 seconds have passed since the scheduler/agent started its
>>> timeout counter, then the agent will restart authentication, causing the
>>> master to discard the previous authentication and start the process all
>>> over again.
>>> What we did in MESOS-2043 is add an increasing backoff to the
>>> authentication timeout, so that we'll eventually wait long enough for an
>>> authentication messages to make it through the queue a few times.
>>>
>>> On Wed, Jul 20, 2016 at 4:17 PM, Douglas Nelson <it...@gmail.com>
>>> wrote:
>>>
>>>> I discovered something else with this issue today. Authentication only
>>>> fails every other time mesos attempts to register an agent node with the
>>>> master node. I've tried completely clearing out zookeeper and starting from
>>>> scratch with the same result. I deploy an application using marathon, kill
>>>> the agent, wait for it to unregister with the master, then start the agent
>>>> again. Repeating that process, every other time it's able to register. When
>>>> it isn't able to register I have to restart the master node before
>>>> registration completes. Could it be related to the app deployed by
>>>> marathon? Has anyone else seen this behavior?
>>>>
>>>> If any other information is needed, let me know. I'm out of ideas about
>>>> the cause.
>>>>
>>>> On Tue, Jul 19, 2016 at 12:47 PM, Douglas Nelson <it...@gmail.com>
>>>> wrote:
>>>>
>>>>> MASTER:
>>>>>
>>>>> Log file created at: 2016/07/19 12:13:55
>>>>> Running on machine: dev1
>>>>> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
>>>>> I0719 12:13:55.437517 11025 logging.cpp:188] INFO level logging
>>>>> started!
>>>>> I0719 12:13:55.438292 11025 main.cpp:237] Build: 2016-07-08 16:57:43
>>>>> by root
>>>>> I0719 12:13:55.438343 11025 main.cpp:239] Version: 0.28.2
>>>>> I0719 12:13:55.438967 11025 main.cpp:260] Using 'HierarchicalDRF'
>>>>> allocator
>>>>> I0719 12:13:55.449280 11025 leveldb.cpp:174] Opened db in 9.870872ms
>>>>> I0719 12:13:55.453482 11025 leveldb.cpp:181] Compacted db in 4.087025ms
>>>>> I0719 12:13:55.453719 11025 leveldb.cpp:196] Created db iterator in
>>>>> 132186ns
>>>>> I0719 12:13:55.453820 11025 leveldb.cpp:202] Seeked to beginning of db
>>>>> in 35181ns
>>>>> I0719 12:13:55.454460 11025 leveldb.cpp:271] Iterated through 3 keys
>>>>> in the db in 592652ns
>>>>> I0719 12:13:55.454795 11025 replica.cpp:779] Replica recovered with
>>>>> log positions 129 -> 130 with 0 holes and 0 unlearned
>>>>> I0719 12:13:55.461314 11049 log.cpp:236] Attempting to join replica to
>>>>> ZooKeeper group
>>>>> I0719 12:13:55.465665 11025 main.cpp:471] Starting Mesos master
>>>>> I0719 12:13:55.478101 11048 recover.cpp:447] Starting replica recovery
>>>>> I0719 12:13:55.482789 11025 master.cpp:375] Master
>>>>> c3c11abc-2f26-479f-8965-db34be5aa567 (dev1.something.com) started on
>>>>> 10.5.7.4:5050
>>>>> I0719 12:13:55.482998 11025 master.cpp:377] Flags at startup:
>>>>> --allocation_interval="1secs" --allocator="HierarchicalDRF"
>>>>> --authenticate="true" --authenticate_http="true"
>>>>> --authenticate_slaves="true" --authenticators="crammd5"
>>>>> --authorizers="local" --credentials="/etc/mesos/auth/credentials"
>>>>> --framework_sorter="drf" --help="false" --hostname_lookup="true"
>>>>> --http_authenticators="basic" --initialize_driver_logging="true"
>>>>> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
>>>>> --logging_level="INFO" --max_completed_frameworks="50"
>>>>> --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
>>>>> --port="5050" --quiet="false" --quorum="1"
>>>>> --recovery_slave_removal_limit="100%" --registry="replicated_log"
>>>>> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
>>>>> --registry_strict="false" --root_submissions="true"
>>>>> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
>>>>> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
>>>>> --work_dir="/var/lib/mesos" --zk="zk://10.5.7.4:2181/mesos"
>>>>> --zk_session_timeout="10secs"
>>>>> I0719 12:13:55.483695 11025 master.cpp:422] Master only allowing
>>>>> authenticated frameworks to register
>>>>> I0719 12:13:55.483753 11025 master.cpp:427] Master only allowing
>>>>> authenticated slaves to register
>>>>> I0719 12:13:55.483783 11025 credentials.hpp:35] Loading credentials
>>>>> for authentication from '/etc/mesos/auth/credentials'
>>>>> W0719 12:13:55.484066 11025 credentials.hpp:50] Permissions on
>>>>> credentials file '/etc/mesos/auth/credentials' are too open. It is
>>>>> recommended that your credentials file is NOT accessible by others.
>>>>> I0719 12:13:55.484253 11048 recover.cpp:473] Replica is in VOTING
>>>>> status
>>>>> I0719 12:13:55.484794 11048 recover.cpp:462] Recover process terminated
>>>>> I0719 12:13:55.494889 11025 master.cpp:467] Using default 'crammd5'
>>>>> authenticator
>>>>> I0719 12:13:55.495386 11025 authenticator.cpp:518] Initializing server
>>>>> SASL
>>>>> I0719 12:13:55.498042 11025 master.cpp:536] Using default 'basic' HTTP
>>>>> authenticator
>>>>> I0719 12:13:55.511752 11025 master.cpp:1650] Successfully attached
>>>>> file '/var/log/mesos/mesos-master.INFO'
>>>>> I0719 12:13:55.512787 11046 contender.cpp:147] Joining the ZK group
>>>>> I0719 12:13:55.603083 11046 group.cpp:349] Group process (group(1)@
>>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>>> I0719 12:13:55.603395 11046 group.cpp:831] Syncing group operations:
>>>>> queue size (joins, cancels, datas) = (0, 0, 0)
>>>>> I0719 12:13:55.603469 11046 group.cpp:427] Trying to create path
>>>>> '/mesos/log_replicas' in ZooKeeper
>>>>> I0719 12:13:55.610569 11051 group.cpp:349] Group process (group(4)@
>>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>>> I0719 12:13:55.610783 11051 group.cpp:831] Syncing group operations:
>>>>> queue size (joins, cancels, datas) = (0, 0, 0)
>>>>> I0719 12:13:55.610844 11051 group.cpp:427] Trying to create path
>>>>> '/mesos' in ZooKeeper
>>>>> I0719 12:13:55.610780 11044 group.cpp:349] Group process (group(3)@
>>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>>> I0719 12:13:55.611119 11044 group.cpp:831] Syncing group operations:
>>>>> queue size (joins, cancels, datas) = (1, 0, 0)
>>>>> I0719 12:13:55.611163 11044 group.cpp:427] Trying to create path
>>>>> '/mesos' in ZooKeeper
>>>>> I0719 12:13:55.611829 11049 group.cpp:349] Group process (group(2)@
>>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>>> I0719 12:13:55.611938 11049 group.cpp:831] Syncing group operations:
>>>>> queue size (joins, cancels, datas) = (1, 0, 0)
>>>>> I0719 12:13:55.611970 11049 group.cpp:427] Trying to create path
>>>>> '/mesos/log_replicas' in ZooKeeper
>>>>> I0719 12:13:55.652545 11048 contender.cpp:263] New candidate (id='2')
>>>>> has entered the contest for leadership
>>>>> I0719 12:13:55.656810 11046 network.hpp:413] ZooKeeper group
>>>>> memberships changed
>>>>> I0719 12:13:55.657207 11044 group.cpp:700] Trying to get
>>>>> '/mesos/log_replicas/0000000001' in ZooKeeper
>>>>> I0719 12:13:55.659225 11049 detector.cpp:152] Detected a new leader:
>>>>> (id='2')
>>>>> I0719 12:13:55.660096 11051 group.cpp:700] Trying to get
>>>>> '/mesos/json.info_0000000002' in ZooKeeper
>>>>> I0719 12:13:55.663472 11050 network.hpp:461] ZooKeeper group PIDs: {
>>>>> log-replica(1)@10.5.7.4:5050 }
>>>>> I0719 12:13:55.664608 11046 detector.cpp:479] A new leading master
>>>>> (UPID=master@10.5.7.4:5050) is detected
>>>>> I0719 12:13:55.665132 11049 master.cpp:1711] The newly elected leader
>>>>> is master@10.5.7.4:5050 with id c3c11abc-2f26-479f-8965-db34be5aa567
>>>>> I0719 12:13:55.665240 11049 master.cpp:1724] Elected as the leading
>>>>> master!
>>>>> I0719 12:13:55.665334 11049 master.cpp:1469] Recovering from registrar
>>>>> I0719 12:13:55.665786 11044 registrar.cpp:307] Recovering registrar
>>>>> I0719 12:13:55.668251 11049 log.cpp:659] Attempting to start the writer
>>>>> I0719 12:13:55.674710 11045 replica.cpp:493] Replica received implicit
>>>>> promise request from (6)@10.5.7.4:5050 with proposal 30
>>>>> I0719 12:13:55.676144 11045 leveldb.cpp:304] Persisting metadata (8
>>>>> bytes) to leveldb took 1.29057ms
>>>>> I0719 12:13:55.676283 11045 replica.cpp:342] Persisted promised to 30
>>>>> I0719 12:13:55.678892 11044 coordinator.cpp:238] Coordinator
>>>>> attempting to fill missing positions
>>>>> I0719 12:13:55.679973 11047 log.cpp:675] Writer started with ending
>>>>> position 130
>>>>> I0719 12:13:55.684237 11051 leveldb.cpp:436] Reading position from
>>>>> leveldb took 180882ns
>>>>> I0719 12:13:55.684692 11051 leveldb.cpp:436] Reading position from
>>>>> leveldb took 86170ns
>>>>> I0719 12:13:55.689906 11045 registrar.cpp:340] Successfully fetched
>>>>> the registry (320B) in 23.838976ms
>>>>> I0719 12:13:55.690568 11045 registrar.cpp:439] Applied 1 operations in
>>>>> 253986ns; attempting to update the 'registry'
>>>>> I0719 12:13:55.695715 11051 log.cpp:683] Attempting to append 359
>>>>> bytes to the log
>>>>> I0719 12:13:55.696043 11045 coordinator.cpp:348] Coordinator
>>>>> attempting to write APPEND action at position 131
>>>>> I0719 12:13:55.698156 11045 replica.cpp:537] Replica received write
>>>>> request for position 131 from (7)@10.5.7.4:5050
>>>>> I0719 12:13:55.699312 11045 leveldb.cpp:341] Persisting action (379
>>>>> bytes) to leveldb took 918900ns
>>>>> I0719 12:13:55.699405 11045 replica.cpp:712] Persisted action at 131
>>>>> I0719 12:13:55.700897 11048 replica.cpp:691] Replica received learned
>>>>> notice for position 131 from @0.0.0.0:0
>>>>> I0719 12:13:55.701294 11048 leveldb.cpp:341] Persisting action (381
>>>>> bytes) to leveldb took 305226ns
>>>>> I0719 12:13:55.701371 11048 replica.cpp:712] Persisted action at 131
>>>>> I0719 12:13:55.701427 11048 replica.cpp:697] Replica learned APPEND
>>>>> action at position 131
>>>>> I0719 12:13:55.705004 11051 registrar.cpp:484] Successfully updated
>>>>> the 'registry' in 14.150144ms
>>>>> I0719 12:13:55.705585 11051 registrar.cpp:370] Successfully recovered
>>>>> registrar
>>>>> I0719 12:13:55.705776 11047 log.cpp:702] Attempting to truncate the
>>>>> log to 131
>>>>> I0719 12:13:55.706045 11046 coordinator.cpp:348] Coordinator
>>>>> attempting to write TRUNCATE action at position 132
>>>>> I0719 12:13:55.707181 11044 replica.cpp:537] Replica received write
>>>>> request for position 132 from (8)@10.5.7.4:5050
>>>>> I0719 12:13:55.707342 11046 master.cpp:1521] Recovered 1 slaves from
>>>>> the Registry (320B) ; allowing 10mins for slaves to re-register
>>>>> I0719 12:13:55.708180 11044 leveldb.cpp:341] Persisting action (18
>>>>> bytes) to leveldb took 889855ns
>>>>> I0719 12:13:55.708283 11044 replica.cpp:712] Persisted action at 132
>>>>> I0719 12:13:55.709445 11051 replica.cpp:691] Replica received learned
>>>>> notice for position 132 from @0.0.0.0:0
>>>>> I0719 12:13:55.709847 11051 leveldb.cpp:341] Persisting action (20
>>>>> bytes) to leveldb took 300975ns
>>>>> I0719 12:13:55.709995 11051 leveldb.cpp:399] Deleting ~2 keys from
>>>>> leveldb took 69236ns
>>>>> I0719 12:13:55.710054 11051 replica.cpp:712] Persisted action at 132
>>>>> I0719 12:13:55.710115 11051 replica.cpp:697] Replica learned TRUNCATE
>>>>> action at position 132
>>>>> I0719 12:14:05.059528 11047 master.cpp:5495] Authenticating
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:05.062243 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:14:05.098544 11049 authenticator.cpp:203] Received SASL
>>>>> authentication start
>>>>> I0719 12:14:05.098788 11049 authenticator.cpp:325] Authentication
>>>>> requires more steps
>>>>> I0719 12:14:05.107225 11044 authenticator.cpp:231] Received SASL
>>>>> authentication step
>>>>> I0719 12:14:05.107561 11044 authenticator.cpp:317] Authentication
>>>>> success
>>>>> I0719 12:14:05.107964 11044 master.cpp:5525] Successfully
>>>>> authenticated principal 'dev' at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:05.110188 11046 master.cpp:2231] Received SUBSCRIBE call
>>>>> for framework 'marathon' at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:05.110599 11046 master.cpp:2302] Subscribing framework
>>>>> marathon with checkpointing enabled and capabilities [  ]
>>>>> I0719 12:14:05.112211 11051 hierarchical.cpp:265] Added framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:14:07.229689 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:14:07.250246 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:14:07.281565 11050 authenticator.cpp:203] Received SASL
>>>>> authentication start
>>>>> I0719 12:14:07.281793 11050 authenticator.cpp:325] Authentication
>>>>> requires more steps
>>>>> I0719 12:14:07.301153 11050 authenticator.cpp:231] Received SASL
>>>>> authentication step
>>>>> I0719 12:14:07.301486 11050 authenticator.cpp:317] Authentication
>>>>> success
>>>>> I0719 12:14:07.302227 11051 master.cpp:5525] Successfully
>>>>> authenticated principal 'dev' at slave(1)@10.5.7.5:5051
>>>>> I0719 12:14:07.307806 11051 master.cpp:4439] Re-registering slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com)
>>>>> I0719 12:14:07.310359 11044 registrar.cpp:439] Applied 1 operations in
>>>>> 176631ns; attempting to update the 'registry'
>>>>> I0719 12:14:07.316319 11049 log.cpp:683] Attempting to append 359
>>>>> bytes to the log
>>>>> I0719 12:14:07.316714 11046 coordinator.cpp:348] Coordinator
>>>>> attempting to write APPEND action at position 133
>>>>> I0719 12:14:07.318164 11046 replica.cpp:537] Replica received write
>>>>> request for position 133 from (11)@10.5.7.4:5050
>>>>> I0719 12:14:07.318792 11046 leveldb.cpp:341] Persisting action (379
>>>>> bytes) to leveldb took 519351ns
>>>>> I0719 12:14:07.318903 11046 replica.cpp:712] Persisted action at 133
>>>>> I0719 12:14:07.320116 11048 replica.cpp:691] Replica received learned
>>>>> notice for position 133 from @0.0.0.0:0
>>>>> I0719 12:14:07.320976 11048 leveldb.cpp:341] Persisting action (381
>>>>> bytes) to leveldb took 785461ns
>>>>> I0719 12:14:07.321053 11048 replica.cpp:712] Persisted action at 133
>>>>> I0719 12:14:07.321106 11048 replica.cpp:697] Replica learned APPEND
>>>>> action at position 133
>>>>> I0719 12:14:07.323272 11046 registrar.cpp:484] Successfully updated
>>>>> the 'registry' in 12.685056ms
>>>>> I0719 12:14:07.323559 11048 log.cpp:702] Attempting to truncate the
>>>>> log to 133
>>>>> I0719 12:14:07.323832 11049 coordinator.cpp:348] Coordinator
>>>>> attempting to write TRUNCATE action at position 134
>>>>> I0719 12:14:07.324877 11049 replica.cpp:537] Replica received write
>>>>> request for position 134 from (12)@10.5.7.4:5050
>>>>> I0719 12:14:07.325728 11049 leveldb.cpp:341] Persisting action (18
>>>>> bytes) to leveldb took 756749ns
>>>>> I0719 12:14:07.325804 11049 replica.cpp:712] Persisted action at 134
>>>>> W0719 12:14:07.326323 11047 master.cpp:6170] Possibly orphaned
>>>>> completed task webapp.42f99194-4dda-11e6-8041-02421568c935 of framework
>>>>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001 that ran on slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com)
>>>>> I0719 12:14:07.326839 11049 replica.cpp:691] Replica received learned
>>>>> notice for position 134 from @0.0.0.0:0
>>>>> I0719 12:14:07.327742 11049 leveldb.cpp:341] Persisting action (20
>>>>> bytes) to leveldb took 717423ns
>>>>> I0719 12:14:07.327713 11047 master.cpp:4521] Re-registered slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com) with cpus(*):4; mem(*):2929; disk(*):242457;
>>>>> ports(*):[31000-32000]
>>>>> I0719 12:14:07.328063 11047 master.cpp:4556] Sending updated
>>>>> checkpointed resources  to slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at
>>>>> slave(1)@10.5.7.5:5051 (dev2.something.com)
>>>>> I0719 12:14:07.328157 11049 leveldb.cpp:399] Deleting ~2 keys from
>>>>> leveldb took 188898ns
>>>>> I0719 12:14:07.328233 11049 replica.cpp:712] Persisted action at 134
>>>>> I0719 12:14:07.328300 11049 replica.cpp:697] Replica learned TRUNCATE
>>>>> action at position 134
>>>>> I0719 12:14:07.329198 11046 hierarchical.cpp:473] Added slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) with
>>>>> cpus(*):4; mem(*):2929; disk(*):242457; ports(*):[31000-32000] (allocated: )
>>>>> I0719 12:14:07.330371 11044 master.cpp:4618] Received update of slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com) with total oversubscribed resources
>>>>> I0719 12:14:07.332751 11046 hierarchical.cpp:531] Slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) updated
>>>>> with oversubscribed resources  (total: cpus(*):4; mem(*):2929;
>>>>> disk(*):242457; ports(*):[31000-32000], allocated: cpus(*):4; mem(*):2929;
>>>>> disk(*):242457; ports(*):[31000-32000])
>>>>> I0719 12:14:07.332794 11047 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:07.432186 11045 master.cpp:3641] Processing DECLINE call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O0 ] for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:09.503839 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:14:09.877415 11044 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:09.877856 11044 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:14:09.880434 11051 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:09.886831 11048 master.cpp:3641] Processing DECLINE call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O1 ] for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:14.894800 11045 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:14.895149 11051 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:14:14.897150 11045 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:14.903162 11044 master.cpp:3641] Processing DECLINE call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O2 ] for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:19.714318 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:14:19.914870 11049 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:19.915210 11047 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:14:19.917094 11049 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:19.923250 11051 master.cpp:3641] Processing DECLINE call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O3 ] for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:20.019886 11050 master.cpp:4997] Performing implicit task
>>>>> state reconciliation for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:29.515525 11048 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:29.515820 11048 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:14:29.517138 11048 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:29.766815 11044 master.cpp:3104] Processing ACCEPT call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O4 ] on slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com) for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:29.771697 11049 master.hpp:177] Adding task
>>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>>>>> mem(*):32; ports(*):[31990-31990] on slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>>>>> I0719 12:14:29.772017 11049 master.cpp:3589] Launching task
>>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 with
>>>>> resources cpus(*):0.1; mem(*):32; ports(*):[31990-31990] on slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com)
>>>>> I0719 12:14:30.589148 11045 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:30.595021 11046 master.cpp:3641] Processing DECLINE call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O5 ] for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:14:30.610872 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:14:30.803971 11048 master.cpp:4763] Status update
>>>>> TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 from slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com)
>>>>> I0719 12:14:30.804188 11048 master.cpp:4811] Forwarding status update
>>>>> TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:14:30.804988 11048 master.cpp:6421] Updating the state of
>>>>> task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_RUNNING,
>>>>> status update state: TASK_RUNNING)
>>>>> I0719 12:14:30.868975 11046 master.cpp:3918] Processing ACKNOWLEDGE
>>>>> call 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f for task
>>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 on
>>>>> slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>>>>> I0719 12:14:40.864478 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:14:51.114728 11050 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:01.365391 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:08.869536 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51290 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:18.887311 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:28.904626 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:38.918462 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:48.932732 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:15:58.946310 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:16:08.961182 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:16:18.976491 11049 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:16:29.148555 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:16:30.951886 11046 master.cpp:5324] Sending 1 offers to
>>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:16:30.958127 11047 master.cpp:3641] Processing DECLINE call
>>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O6 ] for framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:16:39.398538 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:16:49.648846 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:16:59.899797 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:17:07.350857 11047 master.cpp:242] Shutting down slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 due to health check timeout
>>>>> W0719 12:17:07.351191 11047 master.cpp:4930] Shutting down slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com) with message 'health check timed out'
>>>>> I0719 12:17:07.351920 11047 master.cpp:6201] Removing slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com): health check timed out
>>>>> I0719 12:17:07.352468 11047 master.cpp:6421] Updating the state of
>>>>> task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_LOST, status
>>>>> update state: TASK_LOST)
>>>>> I0719 12:17:07.352746 11045 hierarchical.cpp:505] Removed slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>>>>> I0719 12:17:07.353837 11047 master.cpp:6487] Removing task
>>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>>>>> mem(*):32; ports(*):[31990-31990] of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 on slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>>> dev2.something.com)
>>>>> I0719 12:17:07.356307 11046 registrar.cpp:439] Applied 1 operations in
>>>>> 164323ns; attempting to update the 'registry'
>>>>> I0719 12:17:07.360155 11045 log.cpp:683] Attempting to append 184
>>>>> bytes to the log
>>>>> I0719 12:17:07.360399 11050 coordinator.cpp:348] Coordinator
>>>>> attempting to write APPEND action at position 135
>>>>> I0719 12:17:07.361938 11051 replica.cpp:537] Replica received write
>>>>> request for position 135 from (40)@10.5.7.4:5050
>>>>> I0719 12:17:07.362778 11051 leveldb.cpp:341] Persisting action (204
>>>>> bytes) to leveldb took 728456ns
>>>>> I0719 12:17:07.362862 11051 replica.cpp:712] Persisted action at 135
>>>>> I0719 12:17:07.363806 11051 replica.cpp:691] Replica received learned
>>>>> notice for position 135 from @0.0.0.0:0
>>>>> I0719 12:17:07.364334 11051 leveldb.cpp:341] Persisting action (206
>>>>> bytes) to leveldb took 430978ns
>>>>> I0719 12:17:07.364414 11051 replica.cpp:712] Persisted action at 135
>>>>> I0719 12:17:07.364471 11051 replica.cpp:697] Replica learned APPEND
>>>>> action at position 135
>>>>> I0719 12:17:07.366155 11047 registrar.cpp:484] Successfully updated
>>>>> the 'registry' in 9.689088ms
>>>>> I0719 12:17:07.366540 11051 log.cpp:702] Attempting to truncate the
>>>>> log to 135
>>>>> I0719 12:17:07.366657 11046 master.cpp:6320] Removed slave
>>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com): health
>>>>> check timed out
>>>>> I0719 12:17:07.366739 11047 coordinator.cpp:348] Coordinator
>>>>> attempting to write TRUNCATE action at position 136
>>>>> I0719 12:17:07.366789 11046 master.cpp:4806] Sending status update
>>>>> TASK_LOST for task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 'Slave dev2.something.com
>>>>> removed: health check timed out'
>>>>> I0719 12:17:07.367236 11046 master.cpp:6343] Notifying framework
>>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 of lost
>>>>> slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>>>>> after recovering
>>>>> I0719 12:17:07.367993 11049 replica.cpp:537] Replica received write
>>>>> request for position 136 from (41)@10.5.7.4:5050
>>>>> I0719 12:17:07.368701 11049 leveldb.cpp:341] Persisting action (18
>>>>> bytes) to leveldb took 607673ns
>>>>> I0719 12:17:07.368780 11049 replica.cpp:712] Persisted action at 136
>>>>> I0719 12:17:07.369793 11048 replica.cpp:691] Replica received learned
>>>>> notice for position 136 from @0.0.0.0:0
>>>>> I0719 12:17:07.374290 11048 leveldb.cpp:341] Persisting action (20
>>>>> bytes) to leveldb took 4.333673ms
>>>>> I0719 12:17:07.374531 11048 leveldb.cpp:399] Deleting ~2 keys from
>>>>> leveldb took 142498ns
>>>>> I0719 12:17:07.374651 11048 replica.cpp:712] Persisted action at 136
>>>>> I0719 12:17:07.374763 11048 replica.cpp:697] Replica learned TRUNCATE
>>>>> action at position 136
>>>>> I0719 12:17:07.413800 11048 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:17:07.414114 11051 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:17:10.107131 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:17:12.435168 11051 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:17:12.435470 11051 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:17:17.455554 11049 master.cpp:3720] Processing REVIVE call
>>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>>> I0719 12:17:17.456195 11044 hierarchical.cpp:988] Removed offer
>>>>> filters for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>>> I0719 12:17:20.620965 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:17:28.907829 11051 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:28.908550 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:17:30.634392 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:17:33.887810 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:17:33.888640 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:17:33.888864 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:33.889627 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:17:38.890091 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:17:38.890705 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:17:38.890908 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:38.891654 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:17:41.619175 11049 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:17:43.892462 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:17:43.893380 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:17:43.893582 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:43.894261 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:17:48.894907 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:17:48.895597 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:17:48.895742 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:48.896458 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:17:52.622318 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:17:53.897758 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:17:53.898336 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:17:53.898510 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:53.899430 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:17:58.900116 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:17:58.900677 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:17:58.901741 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:17:58.902381 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:03.621628 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:18:03.902536 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:03.903519 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:03.903714 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:03.904594 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:08.904714 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:18:08.905208 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:08.905773 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:08.905913 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:08.906513 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:13.907012 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:18:13.907600 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:13.907958 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:13.908123 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:13.908871 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:14.624475 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:18:18.909356 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:18.910174 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:18.910326 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:18.911255 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:23.911778 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:18:23.912201 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:23.912739 11046 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:23.913128 11046 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:23.913684 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:25.624198 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:18:28.914320 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:28.914964 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:28.915148 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:28.915825 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:33.916498 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:33.917145 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:33.917282 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:33.918520 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:36.625072 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:18:38.918712 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:38.919631 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:38.920052 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:38.920876 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:43.921413 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:43.922125 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:43.922269 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:43.923209 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:47.627984 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:18:48.923180 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:48.923815 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:48.924010 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:48.924589 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:53.925963 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:53.926895 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:53.927124 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:53.927942 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:18:58.626113 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:18:58.927940 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:18:58.928613 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:18:58.928766 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:18:58.930127 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:03.930527 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:03.931396 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:03.931637 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:03.932543 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:08.932302 11048 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:19:08.932852 11048 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:08.933034 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:08.933212 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:08.933835 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:09.627965 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:19:13.934926 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:13.935514 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:13.935757 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:13.936638 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:18.936506 11048 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:19:18.937036 11048 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:18.937420 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:18.937613 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:18.938341 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:20.624949 11049 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:19:23.938432 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:23.939040 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:23.939214 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:23.940078 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:28.941543 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:28.942198 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:28.942435 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:28.943140 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:31.627547 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:19:33.943943 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:19:33.944599 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:33.944835 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:33.944988 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:33.945693 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:38.945986 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:19:38.946478 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:38.946812 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:38.946949 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:38.947438 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:42.628363 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:19:43.948642 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:43.949349 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:43.949550 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:43.950319 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:48.951385 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:48.951884 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:48.952000 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:48.952672 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:53.628947 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:19:53.954244 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:53.954866 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:53.955090 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:53.956058 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:19:58.956197 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:19:58.956797 11046 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:19:58.957006 11046 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:19:58.957918 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:03.958680 11048 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:03.959282 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:03.959450 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:03.960024 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:04.630003 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:20:08.961760 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:20:08.962337 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:08.962745 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:08.962887 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:08.963418 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:13.964138 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:13.964747 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:13.965031 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:13.965772 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:15.630905 11051 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:20:18.966778 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:18.967433 11051 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:18.967687 11051 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:18.968503 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:23.969029 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:23.969832 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:23.969976 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:23.970935 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:26.628504 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:20:28.971153 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:20:28.971578 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:28.971874 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:28.972044 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:28.972594 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:33.973855 11051 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:20:33.974304 11051 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:33.974576 11051 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:33.974689 11051 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:33.975183 11051 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:37.629881 11050 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:20:38.976158 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:38.977018 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:38.977200 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:38.978510 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:43.977927 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:43.978523 11046 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:43.978688 11046 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:43.979673 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:48.631947 11044 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:20:48.980355 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:48.981098 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:48.981258 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:48.982522 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:53.983012 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:53.983757 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:53.983986 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:53.984787 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:58.985410 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:20:58.985873 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:20:58.986018 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:20:58.986199 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:20:58.986820 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:20:59.630745 11049 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:21:03.987254 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:03.987884 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:03.988030 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:03.989013 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:08.989428 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:21:08.990020 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:08.990285 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:08.990468 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:08.990981 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:10.633591 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:21:13.992058 11051 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:13.992918 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:13.993191 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:13.994236 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:18.994022 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:21:18.994477 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:18.994962 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:18.995141 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:18.995689 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:21.647516 11050 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:21:23.996325 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:23.997014 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:23.997248 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:23.998145 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:28.998471 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:28.999652 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:28.999872 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:29.000628 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:32.635963 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:21:34.000746 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:34.001461 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:34.001718 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:34.002413 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:39.003231 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:39.003926 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:39.004137 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:39.004909 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:43.639329 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:21:44.005383 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:44.005898 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:44.006666 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:44.007246 11046 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:49.007254 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:49.007946 11051 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:49.008180 11051 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:49.009217 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:54.009101 11046 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:54.009840 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:54.010089 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:54.010777 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:21:54.634322 11050 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:21:59.011060 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:21:59.011662 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:21:59.011852 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:21:59.012444 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:04.013264 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:22:04.013864 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:04.014137 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:04.014339 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:04.015051 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:05.633569 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:22:09.015625 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:09.016332 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:09.016549 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:09.017400 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:14.018102 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:14.018810 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:14.018975 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:14.020356 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:16.638113 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:22:19.020917 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:22:19.021430 11049 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:19.021756 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:19.021895 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:19.022616 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:24.023571 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:24.024431 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:24.024754 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:24.026060 11050 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:27.643009 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:22:29.026162 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:29.026885 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:29.027034 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:29.028298 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:34.028923 11050 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:34.029616 11050 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:34.029865 11050 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:34.030767 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:38.638548 11045 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:22:39.031050 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:39.031689 11046 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:39.031838 11046 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:39.032711 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:44.033393 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:44.034443 11046 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:44.034685 11046 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:44.035321 11051 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:49.035828 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:49.036345 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:49.036553 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:49.037240 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:49.636526 11047 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:22:54.038525 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:22:54.039108 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:54.039549 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:54.039691 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:54.040293 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:22:59.040702 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:22:59.041216 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:22:59.041524 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:22:59.041749 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:22:59.042407 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:00.639822 11049 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:23:04.042881 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:04.043432 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:04.043550 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:04.044637 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> W0719 12:23:09.044903 11051 master.cpp:5541] Authentication timed out
>>>>> W0719 12:23:09.045856 11049 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:09.046164 11049 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:09.046769 11049 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:11.640506 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:23:14.047653 11048 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:14.048249 11048 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:14.048429 11048 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:14.049170 11048 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:19.049494 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:19.050199 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:19.050398 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:19.051105 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:22.639598 11046 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:23:24.051370 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:24.052043 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:24.052255 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:24.053045 11045 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:29.053254 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:23:29.053788 11044 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:29.054085 11044 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:29.054247 11044 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:29.054867 11044 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:33.639268 11048 http.cpp:312] HTTP GET for /master/state
>>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>>> Safari/537.36'
>>>>> I0719 12:23:34.055438 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> I0719 12:23:34.055974 11047 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:34.056213 11047 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:34.056349 11047 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:34.056926 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:39.057729 11045 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10.5.7.5:5051 because authentication is still
>>>>> in progress
>>>>> W0719 12:23:39.058291 11045 master.cpp:5522] Failed to authenticate
>>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>> I0719 12:23:39.058435 11045 master.cpp:5495] Authenticating slave(1)@
>>>>> 10.5.7.5:5051
>>>>> I0719 12:23:39.059099 11047 authenticator.cpp:98] Creating new server
>>>>> SASL connection
>>>>> I0719 12:23:44.059839 11051 master.cpp:5481] Queuing up authentication
>>>>> request from slave(1)@10 <http://10.5.7.5:5051>
>>>>>
>>>> ...
>>>
>>> [Message clipped]
>>
>>
>>
>

Re: Possible authentication bug

Posted by Adam Bordelon <ad...@mesosphere.io>.

The flag is new in 1.0.0, so we'd have to backport it for 0.28 for you, or
you'd have to try out 1.0 from master or an rc.

On Thu, Jul 21, 2016 at 11:26 AM, Douglas Nelson <it...@gmail.com>
wrote:

> I tried setting --authentication_backoff_factor=9secs to experiment with a
> different time. The mesos agent doesn't start at all when I set that flag?
> I'm running mesos 0.28.2.
>
> On Wed, Jul 20, 2016 at 9:12 PM, Adam Bordelon <ad...@mesosphere.io> wrote:
>
>> This does indeed look identical to the logs from previous occurrences of
>> https://issues.apache.org/jira/browse/MESOS-2043
>> This can happen when your master's message queue gets clogged (check
>> /metrics/snapshot). The scheduler/agent begins authentication and starts
>> its timeout counter. Eventually, the master gets the message out of its
>> queue and asks the scheduler/agent for the next step in authentication. If
>> more than 5 seconds have passed since the scheduler/agent started its
>> timeout counter, then the agent will restart authentication, causing the
>> master to discard the previous authentication and start the process all
>> over again.
>> What we did in MESOS-2043 is add an increasing backoff to the
>> authentication timeout, so that we'll eventually wait long enough for an
>> authentication messages to make it through the queue a few times.
>>
>> On Wed, Jul 20, 2016 at 4:17 PM, Douglas Nelson <it...@gmail.com>
>> wrote:
>>
>>> I discovered something else with this issue today. Authentication only
>>> fails every other time mesos attempts to register an agent node with the
>>> master node. I've tried completely clearing out zookeeper and starting from
>>> scratch with the same result. I deploy an application using marathon, kill
>>> the agent, wait for it to unregister with the master, then start the agent
>>> again. Repeating that process, every other time it's able to register. When
>>> it isn't able to register I have to restart the master node before
>>> registration completes. Could it be related to the app deployed by
>>> marathon? Has anyone else seen this behavior?
>>>
>>> If any other information is needed, let me know. I'm out of ideas about
>>> the cause.
>>>
>>> On Tue, Jul 19, 2016 at 12:47 PM, Douglas Nelson <it...@gmail.com>
>>> wrote:
>>>
>>>> MASTER:
>>>>
>>>> Log file created at: 2016/07/19 12:13:55
>>>> Running on machine: dev1
>>>> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
>>>> I0719 12:13:55.437517 11025 logging.cpp:188] INFO level logging started!
>>>> I0719 12:13:55.438292 11025 main.cpp:237] Build: 2016-07-08 16:57:43 by
>>>> root
>>>> I0719 12:13:55.438343 11025 main.cpp:239] Version: 0.28.2
>>>> I0719 12:13:55.438967 11025 main.cpp:260] Using 'HierarchicalDRF'
>>>> allocator
>>>> I0719 12:13:55.449280 11025 leveldb.cpp:174] Opened db in 9.870872ms
>>>> I0719 12:13:55.453482 11025 leveldb.cpp:181] Compacted db in 4.087025ms
>>>> I0719 12:13:55.453719 11025 leveldb.cpp:196] Created db iterator in
>>>> 132186ns
>>>> I0719 12:13:55.453820 11025 leveldb.cpp:202] Seeked to beginning of db
>>>> in 35181ns
>>>> I0719 12:13:55.454460 11025 leveldb.cpp:271] Iterated through 3 keys in
>>>> the db in 592652ns
>>>> I0719 12:13:55.454795 11025 replica.cpp:779] Replica recovered with log
>>>> positions 129 -> 130 with 0 holes and 0 unlearned
>>>> I0719 12:13:55.461314 11049 log.cpp:236] Attempting to join replica to
>>>> ZooKeeper group
>>>> I0719 12:13:55.465665 11025 main.cpp:471] Starting Mesos master
>>>> I0719 12:13:55.478101 11048 recover.cpp:447] Starting replica recovery
>>>> I0719 12:13:55.482789 11025 master.cpp:375] Master
>>>> c3c11abc-2f26-479f-8965-db34be5aa567 (dev1.something.com) started on
>>>> 10.5.7.4:5050
>>>> I0719 12:13:55.482998 11025 master.cpp:377] Flags at startup:
>>>> --allocation_interval="1secs" --allocator="HierarchicalDRF"
>>>> --authenticate="true" --authenticate_http="true"
>>>> --authenticate_slaves="true" --authenticators="crammd5"
>>>> --authorizers="local" --credentials="/etc/mesos/auth/credentials"
>>>> --framework_sorter="drf" --help="false" --hostname_lookup="true"
>>>> --http_authenticators="basic" --initialize_driver_logging="true"
>>>> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
>>>> --logging_level="INFO" --max_completed_frameworks="50"
>>>> --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
>>>> --port="5050" --quiet="false" --quorum="1"
>>>> --recovery_slave_removal_limit="100%" --registry="replicated_log"
>>>> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
>>>> --registry_strict="false" --root_submissions="true"
>>>> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
>>>> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
>>>> --work_dir="/var/lib/mesos" --zk="zk://10.5.7.4:2181/mesos"
>>>> --zk_session_timeout="10secs"
>>>> I0719 12:13:55.483695 11025 master.cpp:422] Master only allowing
>>>> authenticated frameworks to register
>>>> I0719 12:13:55.483753 11025 master.cpp:427] Master only allowing
>>>> authenticated slaves to register
>>>> I0719 12:13:55.483783 11025 credentials.hpp:35] Loading credentials for
>>>> authentication from '/etc/mesos/auth/credentials'
>>>> W0719 12:13:55.484066 11025 credentials.hpp:50] Permissions on
>>>> credentials file '/etc/mesos/auth/credentials' are too open. It is
>>>> recommended that your credentials file is NOT accessible by others.
>>>> I0719 12:13:55.484253 11048 recover.cpp:473] Replica is in VOTING status
>>>> I0719 12:13:55.484794 11048 recover.cpp:462] Recover process terminated
>>>> I0719 12:13:55.494889 11025 master.cpp:467] Using default 'crammd5'
>>>> authenticator
>>>> I0719 12:13:55.495386 11025 authenticator.cpp:518] Initializing server
>>>> SASL
>>>> I0719 12:13:55.498042 11025 master.cpp:536] Using default 'basic' HTTP
>>>> authenticator
>>>> I0719 12:13:55.511752 11025 master.cpp:1650] Successfully attached file
>>>> '/var/log/mesos/mesos-master.INFO'
>>>> I0719 12:13:55.512787 11046 contender.cpp:147] Joining the ZK group
>>>> I0719 12:13:55.603083 11046 group.cpp:349] Group process (group(1)@
>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>> I0719 12:13:55.603395 11046 group.cpp:831] Syncing group operations:
>>>> queue size (joins, cancels, datas) = (0, 0, 0)
>>>> I0719 12:13:55.603469 11046 group.cpp:427] Trying to create path
>>>> '/mesos/log_replicas' in ZooKeeper
>>>> I0719 12:13:55.610569 11051 group.cpp:349] Group process (group(4)@
>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>> I0719 12:13:55.610783 11051 group.cpp:831] Syncing group operations:
>>>> queue size (joins, cancels, datas) = (0, 0, 0)
>>>> I0719 12:13:55.610844 11051 group.cpp:427] Trying to create path
>>>> '/mesos' in ZooKeeper
>>>> I0719 12:13:55.610780 11044 group.cpp:349] Group process (group(3)@
>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>> I0719 12:13:55.611119 11044 group.cpp:831] Syncing group operations:
>>>> queue size (joins, cancels, datas) = (1, 0, 0)
>>>> I0719 12:13:55.611163 11044 group.cpp:427] Trying to create path
>>>> '/mesos' in ZooKeeper
>>>> I0719 12:13:55.611829 11049 group.cpp:349] Group process (group(2)@
>>>> 10.5.7.4:5050) connected to ZooKeeper
>>>> I0719 12:13:55.611938 11049 group.cpp:831] Syncing group operations:
>>>> queue size (joins, cancels, datas) = (1, 0, 0)
>>>> I0719 12:13:55.611970 11049 group.cpp:427] Trying to create path
>>>> '/mesos/log_replicas' in ZooKeeper
>>>> I0719 12:13:55.652545 11048 contender.cpp:263] New candidate (id='2')
>>>> has entered the contest for leadership
>>>> I0719 12:13:55.656810 11046 network.hpp:413] ZooKeeper group
>>>> memberships changed
>>>> I0719 12:13:55.657207 11044 group.cpp:700] Trying to get
>>>> '/mesos/log_replicas/0000000001' in ZooKeeper
>>>> I0719 12:13:55.659225 11049 detector.cpp:152] Detected a new leader:
>>>> (id='2')
>>>> I0719 12:13:55.660096 11051 group.cpp:700] Trying to get
>>>> '/mesos/json.info_0000000002' in ZooKeeper
>>>> I0719 12:13:55.663472 11050 network.hpp:461] ZooKeeper group PIDs: {
>>>> log-replica(1)@10.5.7.4:5050 }
>>>> I0719 12:13:55.664608 11046 detector.cpp:479] A new leading master
>>>> (UPID=master@10.5.7.4:5050) is detected
>>>> I0719 12:13:55.665132 11049 master.cpp:1711] The newly elected leader
>>>> is master@10.5.7.4:5050 with id c3c11abc-2f26-479f-8965-db34be5aa567
>>>> I0719 12:13:55.665240 11049 master.cpp:1724] Elected as the leading
>>>> master!
>>>> I0719 12:13:55.665334 11049 master.cpp:1469] Recovering from registrar
>>>> I0719 12:13:55.665786 11044 registrar.cpp:307] Recovering registrar
>>>> I0719 12:13:55.668251 11049 log.cpp:659] Attempting to start the writer
>>>> I0719 12:13:55.674710 11045 replica.cpp:493] Replica received implicit
>>>> promise request from (6)@10.5.7.4:5050 with proposal 30
>>>> I0719 12:13:55.676144 11045 leveldb.cpp:304] Persisting metadata (8
>>>> bytes) to leveldb took 1.29057ms
>>>> I0719 12:13:55.676283 11045 replica.cpp:342] Persisted promised to 30
>>>> I0719 12:13:55.678892 11044 coordinator.cpp:238] Coordinator attempting
>>>> to fill missing positions
>>>> I0719 12:13:55.679973 11047 log.cpp:675] Writer started with ending
>>>> position 130
>>>> I0719 12:13:55.684237 11051 leveldb.cpp:436] Reading position from
>>>> leveldb took 180882ns
>>>> I0719 12:13:55.684692 11051 leveldb.cpp:436] Reading position from
>>>> leveldb took 86170ns
>>>> I0719 12:13:55.689906 11045 registrar.cpp:340] Successfully fetched the
>>>> registry (320B) in 23.838976ms
>>>> I0719 12:13:55.690568 11045 registrar.cpp:439] Applied 1 operations in
>>>> 253986ns; attempting to update the 'registry'
>>>> I0719 12:13:55.695715 11051 log.cpp:683] Attempting to append 359 bytes
>>>> to the log
>>>> I0719 12:13:55.696043 11045 coordinator.cpp:348] Coordinator attempting
>>>> to write APPEND action at position 131
>>>> I0719 12:13:55.698156 11045 replica.cpp:537] Replica received write
>>>> request for position 131 from (7)@10.5.7.4:5050
>>>> I0719 12:13:55.699312 11045 leveldb.cpp:341] Persisting action (379
>>>> bytes) to leveldb took 918900ns
>>>> I0719 12:13:55.699405 11045 replica.cpp:712] Persisted action at 131
>>>> I0719 12:13:55.700897 11048 replica.cpp:691] Replica received learned
>>>> notice for position 131 from @0.0.0.0:0
>>>> I0719 12:13:55.701294 11048 leveldb.cpp:341] Persisting action (381
>>>> bytes) to leveldb took 305226ns
>>>> I0719 12:13:55.701371 11048 replica.cpp:712] Persisted action at 131
>>>> I0719 12:13:55.701427 11048 replica.cpp:697] Replica learned APPEND
>>>> action at position 131
>>>> I0719 12:13:55.705004 11051 registrar.cpp:484] Successfully updated the
>>>> 'registry' in 14.150144ms
>>>> I0719 12:13:55.705585 11051 registrar.cpp:370] Successfully recovered
>>>> registrar
>>>> I0719 12:13:55.705776 11047 log.cpp:702] Attempting to truncate the log
>>>> to 131
>>>> I0719 12:13:55.706045 11046 coordinator.cpp:348] Coordinator attempting
>>>> to write TRUNCATE action at position 132
>>>> I0719 12:13:55.707181 11044 replica.cpp:537] Replica received write
>>>> request for position 132 from (8)@10.5.7.4:5050
>>>> I0719 12:13:55.707342 11046 master.cpp:1521] Recovered 1 slaves from
>>>> the Registry (320B) ; allowing 10mins for slaves to re-register
>>>> I0719 12:13:55.708180 11044 leveldb.cpp:341] Persisting action (18
>>>> bytes) to leveldb took 889855ns
>>>> I0719 12:13:55.708283 11044 replica.cpp:712] Persisted action at 132
>>>> I0719 12:13:55.709445 11051 replica.cpp:691] Replica received learned
>>>> notice for position 132 from @0.0.0.0:0
>>>> I0719 12:13:55.709847 11051 leveldb.cpp:341] Persisting action (20
>>>> bytes) to leveldb took 300975ns
>>>> I0719 12:13:55.709995 11051 leveldb.cpp:399] Deleting ~2 keys from
>>>> leveldb took 69236ns
>>>> I0719 12:13:55.710054 11051 replica.cpp:712] Persisted action at 132
>>>> I0719 12:13:55.710115 11051 replica.cpp:697] Replica learned TRUNCATE
>>>> action at position 132
>>>> I0719 12:14:05.059528 11047 master.cpp:5495] Authenticating
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:05.062243 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:14:05.098544 11049 authenticator.cpp:203] Received SASL
>>>> authentication start
>>>> I0719 12:14:05.098788 11049 authenticator.cpp:325] Authentication
>>>> requires more steps
>>>> I0719 12:14:05.107225 11044 authenticator.cpp:231] Received SASL
>>>> authentication step
>>>> I0719 12:14:05.107561 11044 authenticator.cpp:317] Authentication
>>>> success
>>>> I0719 12:14:05.107964 11044 master.cpp:5525] Successfully authenticated
>>>> principal 'dev' at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:05.110188 11046 master.cpp:2231] Received SUBSCRIBE call
>>>> for framework 'marathon' at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:05.110599 11046 master.cpp:2302] Subscribing framework
>>>> marathon with checkpointing enabled and capabilities [  ]
>>>> I0719 12:14:05.112211 11051 hierarchical.cpp:265] Added framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:14:07.229689 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:14:07.250246 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:14:07.281565 11050 authenticator.cpp:203] Received SASL
>>>> authentication start
>>>> I0719 12:14:07.281793 11050 authenticator.cpp:325] Authentication
>>>> requires more steps
>>>> I0719 12:14:07.301153 11050 authenticator.cpp:231] Received SASL
>>>> authentication step
>>>> I0719 12:14:07.301486 11050 authenticator.cpp:317] Authentication
>>>> success
>>>> I0719 12:14:07.302227 11051 master.cpp:5525] Successfully authenticated
>>>> principal 'dev' at slave(1)@10.5.7.5:5051
>>>> I0719 12:14:07.307806 11051 master.cpp:4439] Re-registering slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com)
>>>> I0719 12:14:07.310359 11044 registrar.cpp:439] Applied 1 operations in
>>>> 176631ns; attempting to update the 'registry'
>>>> I0719 12:14:07.316319 11049 log.cpp:683] Attempting to append 359 bytes
>>>> to the log
>>>> I0719 12:14:07.316714 11046 coordinator.cpp:348] Coordinator attempting
>>>> to write APPEND action at position 133
>>>> I0719 12:14:07.318164 11046 replica.cpp:537] Replica received write
>>>> request for position 133 from (11)@10.5.7.4:5050
>>>> I0719 12:14:07.318792 11046 leveldb.cpp:341] Persisting action (379
>>>> bytes) to leveldb took 519351ns
>>>> I0719 12:14:07.318903 11046 replica.cpp:712] Persisted action at 133
>>>> I0719 12:14:07.320116 11048 replica.cpp:691] Replica received learned
>>>> notice for position 133 from @0.0.0.0:0
>>>> I0719 12:14:07.320976 11048 leveldb.cpp:341] Persisting action (381
>>>> bytes) to leveldb took 785461ns
>>>> I0719 12:14:07.321053 11048 replica.cpp:712] Persisted action at 133
>>>> I0719 12:14:07.321106 11048 replica.cpp:697] Replica learned APPEND
>>>> action at position 133
>>>> I0719 12:14:07.323272 11046 registrar.cpp:484] Successfully updated the
>>>> 'registry' in 12.685056ms
>>>> I0719 12:14:07.323559 11048 log.cpp:702] Attempting to truncate the log
>>>> to 133
>>>> I0719 12:14:07.323832 11049 coordinator.cpp:348] Coordinator attempting
>>>> to write TRUNCATE action at position 134
>>>> I0719 12:14:07.324877 11049 replica.cpp:537] Replica received write
>>>> request for position 134 from (12)@10.5.7.4:5050
>>>> I0719 12:14:07.325728 11049 leveldb.cpp:341] Persisting action (18
>>>> bytes) to leveldb took 756749ns
>>>> I0719 12:14:07.325804 11049 replica.cpp:712] Persisted action at 134
>>>> W0719 12:14:07.326323 11047 master.cpp:6170] Possibly orphaned
>>>> completed task webapp.42f99194-4dda-11e6-8041-02421568c935 of framework
>>>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001 that ran on slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com)
>>>> I0719 12:14:07.326839 11049 replica.cpp:691] Replica received learned
>>>> notice for position 134 from @0.0.0.0:0
>>>> I0719 12:14:07.327742 11049 leveldb.cpp:341] Persisting action (20
>>>> bytes) to leveldb took 717423ns
>>>> I0719 12:14:07.327713 11047 master.cpp:4521] Re-registered slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com) with cpus(*):4; mem(*):2929; disk(*):242457;
>>>> ports(*):[31000-32000]
>>>> I0719 12:14:07.328063 11047 master.cpp:4556] Sending updated
>>>> checkpointed resources  to slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at
>>>> slave(1)@10.5.7.5:5051 (dev2.something.com)
>>>> I0719 12:14:07.328157 11049 leveldb.cpp:399] Deleting ~2 keys from
>>>> leveldb took 188898ns
>>>> I0719 12:14:07.328233 11049 replica.cpp:712] Persisted action at 134
>>>> I0719 12:14:07.328300 11049 replica.cpp:697] Replica learned TRUNCATE
>>>> action at position 134
>>>> I0719 12:14:07.329198 11046 hierarchical.cpp:473] Added slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) with
>>>> cpus(*):4; mem(*):2929; disk(*):242457; ports(*):[31000-32000] (allocated: )
>>>> I0719 12:14:07.330371 11044 master.cpp:4618] Received update of slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com) with total oversubscribed resources
>>>> I0719 12:14:07.332751 11046 hierarchical.cpp:531] Slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) updated
>>>> with oversubscribed resources  (total: cpus(*):4; mem(*):2929;
>>>> disk(*):242457; ports(*):[31000-32000], allocated: cpus(*):4; mem(*):2929;
>>>> disk(*):242457; ports(*):[31000-32000])
>>>> I0719 12:14:07.332794 11047 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:07.432186 11045 master.cpp:3641] Processing DECLINE call
>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O0 ] for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:09.503839 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:14:09.877415 11044 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:09.877856 11044 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:14:09.880434 11051 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:09.886831 11048 master.cpp:3641] Processing DECLINE call
>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O1 ] for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:14.894800 11045 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:14.895149 11051 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:14:14.897150 11045 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:14.903162 11044 master.cpp:3641] Processing DECLINE call
>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O2 ] for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:19.714318 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:14:19.914870 11049 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:19.915210 11047 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:14:19.917094 11049 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:19.923250 11051 master.cpp:3641] Processing DECLINE call
>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O3 ] for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:20.019886 11050 master.cpp:4997] Performing implicit task
>>>> state reconciliation for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:29.515525 11048 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:29.515820 11048 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:14:29.517138 11048 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:29.766815 11044 master.cpp:3104] Processing ACCEPT call for
>>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O4 ] on slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com) for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:29.771697 11049 master.hpp:177] Adding task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>>>> mem(*):32; ports(*):[31990-31990] on slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>>>> I0719 12:14:29.772017 11049 master.cpp:3589] Launching task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 with
>>>> resources cpus(*):0.1; mem(*):32; ports(*):[31990-31990] on slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com)
>>>> I0719 12:14:30.589148 11045 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:30.595021 11046 master.cpp:3641] Processing DECLINE call
>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O5 ] for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:14:30.610872 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:14:30.803971 11048 master.cpp:4763] Status update TASK_RUNNING
>>>> (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 from slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com)
>>>> I0719 12:14:30.804188 11048 master.cpp:4811] Forwarding status update
>>>> TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:14:30.804988 11048 master.cpp:6421] Updating the state of task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_RUNNING,
>>>> status update state: TASK_RUNNING)
>>>> I0719 12:14:30.868975 11046 master.cpp:3918] Processing ACKNOWLEDGE
>>>> call 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f for task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 on slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>>>> I0719 12:14:40.864478 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:14:51.114728 11050 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:01.365391 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:08.869536 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51290 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:18.887311 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:28.904626 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:38.918462 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:48.932732 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:15:58.946310 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:16:08.961182 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:16:18.976491 11049 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:16:29.148555 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:16:30.951886 11046 master.cpp:5324] Sending 1 offers to
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:16:30.958127 11047 master.cpp:3641] Processing DECLINE call
>>>> for offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O6 ] for framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:16:39.398538 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:16:49.648846 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:16:59.899797 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:17:07.350857 11047 master.cpp:242] Shutting down slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 due to health check timeout
>>>> W0719 12:17:07.351191 11047 master.cpp:4930] Shutting down slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com) with message 'health check timed out'
>>>> I0719 12:17:07.351920 11047 master.cpp:6201] Removing slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com): health check timed out
>>>> I0719 12:17:07.352468 11047 master.cpp:6421] Updating the state of task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_LOST, status
>>>> update state: TASK_LOST)
>>>> I0719 12:17:07.352746 11045 hierarchical.cpp:505] Removed slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>>>> I0719 12:17:07.353837 11047 master.cpp:6487] Removing task
>>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>>>> mem(*):32; ports(*):[31990-31990] of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 on slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>>> dev2.something.com)
>>>> I0719 12:17:07.356307 11046 registrar.cpp:439] Applied 1 operations in
>>>> 164323ns; attempting to update the 'registry'
>>>> I0719 12:17:07.360155 11045 log.cpp:683] Attempting to append 184 bytes
>>>> to the log
>>>> I0719 12:17:07.360399 11050 coordinator.cpp:348] Coordinator attempting
>>>> to write APPEND action at position 135
>>>> I0719 12:17:07.361938 11051 replica.cpp:537] Replica received write
>>>> request for position 135 from (40)@10.5.7.4:5050
>>>> I0719 12:17:07.362778 11051 leveldb.cpp:341] Persisting action (204
>>>> bytes) to leveldb took 728456ns
>>>> I0719 12:17:07.362862 11051 replica.cpp:712] Persisted action at 135
>>>> I0719 12:17:07.363806 11051 replica.cpp:691] Replica received learned
>>>> notice for position 135 from @0.0.0.0:0
>>>> I0719 12:17:07.364334 11051 leveldb.cpp:341] Persisting action (206
>>>> bytes) to leveldb took 430978ns
>>>> I0719 12:17:07.364414 11051 replica.cpp:712] Persisted action at 135
>>>> I0719 12:17:07.364471 11051 replica.cpp:697] Replica learned APPEND
>>>> action at position 135
>>>> I0719 12:17:07.366155 11047 registrar.cpp:484] Successfully updated the
>>>> 'registry' in 9.689088ms
>>>> I0719 12:17:07.366540 11051 log.cpp:702] Attempting to truncate the log
>>>> to 135
>>>> I0719 12:17:07.366657 11046 master.cpp:6320] Removed slave
>>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com): health
>>>> check timed out
>>>> I0719 12:17:07.366739 11047 coordinator.cpp:348] Coordinator attempting
>>>> to write TRUNCATE action at position 136
>>>> I0719 12:17:07.366789 11046 master.cpp:4806] Sending status update
>>>> TASK_LOST for task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 'Slave dev2.something.com
>>>> removed: health check timed out'
>>>> I0719 12:17:07.367236 11046 master.cpp:6343] Notifying framework
>>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 of lost
>>>> slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>>>> after recovering
>>>> I0719 12:17:07.367993 11049 replica.cpp:537] Replica received write
>>>> request for position 136 from (41)@10.5.7.4:5050
>>>> I0719 12:17:07.368701 11049 leveldb.cpp:341] Persisting action (18
>>>> bytes) to leveldb took 607673ns
>>>> I0719 12:17:07.368780 11049 replica.cpp:712] Persisted action at 136
>>>> I0719 12:17:07.369793 11048 replica.cpp:691] Replica received learned
>>>> notice for position 136 from @0.0.0.0:0
>>>> I0719 12:17:07.374290 11048 leveldb.cpp:341] Persisting action (20
>>>> bytes) to leveldb took 4.333673ms
>>>> I0719 12:17:07.374531 11048 leveldb.cpp:399] Deleting ~2 keys from
>>>> leveldb took 142498ns
>>>> I0719 12:17:07.374651 11048 replica.cpp:712] Persisted action at 136
>>>> I0719 12:17:07.374763 11048 replica.cpp:697] Replica learned TRUNCATE
>>>> action at position 136
>>>> I0719 12:17:07.413800 11048 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:17:07.414114 11051 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:17:10.107131 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:17:12.435168 11051 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:17:12.435470 11051 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:17:17.455554 11049 master.cpp:3720] Processing REVIVE call for
>>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>>> I0719 12:17:17.456195 11044 hierarchical.cpp:988] Removed offer filters
>>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>>> I0719 12:17:20.620965 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:17:28.907829 11051 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:28.908550 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:17:30.634392 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:17:33.887810 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:17:33.888640 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:17:33.888864 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:33.889627 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:17:38.890091 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:17:38.890705 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:17:38.890908 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:38.891654 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:17:41.619175 11049 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:17:43.892462 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:17:43.893380 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:17:43.893582 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:43.894261 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:17:48.894907 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:17:48.895597 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:17:48.895742 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:48.896458 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:17:52.622318 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:17:53.897758 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:17:53.898336 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:17:53.898510 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:53.899430 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:17:58.900116 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:17:58.900677 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:17:58.901741 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:17:58.902381 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:03.621628 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:18:03.902536 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:03.903519 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:03.903714 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:03.904594 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:08.904714 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:18:08.905208 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:08.905773 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:08.905913 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:08.906513 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:13.907012 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:18:13.907600 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:13.907958 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:13.908123 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:13.908871 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:14.624475 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:18:18.909356 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:18.910174 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:18.910326 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:18.911255 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:23.911778 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:18:23.912201 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:23.912739 11046 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:23.913128 11046 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:23.913684 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:25.624198 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:18:28.914320 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:28.914964 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:28.915148 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:28.915825 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:33.916498 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:33.917145 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:33.917282 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:33.918520 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:36.625072 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:18:38.918712 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:38.919631 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:38.920052 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:38.920876 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:43.921413 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:43.922125 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:43.922269 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:43.923209 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:47.627984 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:18:48.923180 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:48.923815 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:48.924010 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:48.924589 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:53.925963 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:53.926895 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:53.927124 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:53.927942 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:18:58.626113 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:18:58.927940 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:18:58.928613 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:18:58.928766 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:18:58.930127 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:03.930527 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:03.931396 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:03.931637 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:03.932543 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:08.932302 11048 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:19:08.932852 11048 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:08.933034 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:08.933212 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:08.933835 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:09.627965 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:19:13.934926 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:13.935514 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:13.935757 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:13.936638 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:18.936506 11048 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:19:18.937036 11048 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:18.937420 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:18.937613 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:18.938341 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:20.624949 11049 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:19:23.938432 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:23.939040 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:23.939214 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:23.940078 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:28.941543 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:28.942198 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:28.942435 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:28.943140 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:31.627547 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:19:33.943943 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:19:33.944599 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:33.944835 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:33.944988 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:33.945693 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:38.945986 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:19:38.946478 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:38.946812 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:38.946949 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:38.947438 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:42.628363 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:19:43.948642 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:43.949349 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:43.949550 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:43.950319 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:48.951385 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:48.951884 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:48.952000 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:48.952672 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:53.628947 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:19:53.954244 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:53.954866 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:53.955090 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:53.956058 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:19:58.956197 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:19:58.956797 11046 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:19:58.957006 11046 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:19:58.957918 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:03.958680 11048 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:03.959282 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:03.959450 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:03.960024 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:04.630003 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:20:08.961760 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:20:08.962337 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:08.962745 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:08.962887 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:08.963418 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:13.964138 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:13.964747 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:13.965031 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:13.965772 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:15.630905 11051 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:20:18.966778 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:18.967433 11051 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:18.967687 11051 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:18.968503 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:23.969029 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:23.969832 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:23.969976 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:23.970935 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:26.628504 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:20:28.971153 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:20:28.971578 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:28.971874 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:28.972044 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:28.972594 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:33.973855 11051 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:20:33.974304 11051 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:33.974576 11051 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:33.974689 11051 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:33.975183 11051 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:37.629881 11050 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:20:38.976158 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:38.977018 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:38.977200 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:38.978510 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:43.977927 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:43.978523 11046 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:43.978688 11046 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:43.979673 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:48.631947 11044 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:20:48.980355 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:48.981098 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:48.981258 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:48.982522 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:53.983012 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:53.983757 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:53.983986 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:53.984787 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:58.985410 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:20:58.985873 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:20:58.986018 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:20:58.986199 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:20:58.986820 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:20:59.630745 11049 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:21:03.987254 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:03.987884 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:03.988030 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:03.989013 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:08.989428 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:21:08.990020 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:08.990285 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:08.990468 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:08.990981 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:10.633591 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:21:13.992058 11051 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:13.992918 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:13.993191 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:13.994236 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:18.994022 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:21:18.994477 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:18.994962 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:18.995141 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:18.995689 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:21.647516 11050 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:21:23.996325 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:23.997014 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:23.997248 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:23.998145 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:28.998471 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:28.999652 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:28.999872 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:29.000628 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:32.635963 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:21:34.000746 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:34.001461 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:34.001718 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:34.002413 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:39.003231 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:39.003926 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:39.004137 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:39.004909 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:43.639329 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:21:44.005383 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:44.005898 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:44.006666 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:44.007246 11046 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:49.007254 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:49.007946 11051 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:49.008180 11051 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:49.009217 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:54.009101 11046 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:54.009840 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:54.010089 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:54.010777 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:21:54.634322 11050 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:21:59.011060 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:21:59.011662 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:21:59.011852 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:21:59.012444 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:04.013264 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:22:04.013864 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:04.014137 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:04.014339 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:04.015051 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:05.633569 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:22:09.015625 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:09.016332 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:09.016549 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:09.017400 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:14.018102 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:14.018810 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:14.018975 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:14.020356 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:16.638113 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:22:19.020917 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:22:19.021430 11049 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:19.021756 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:19.021895 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:19.022616 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:24.023571 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:24.024431 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:24.024754 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:24.026060 11050 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:27.643009 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:22:29.026162 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:29.026885 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:29.027034 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:29.028298 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:34.028923 11050 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:34.029616 11050 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:34.029865 11050 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:34.030767 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:38.638548 11045 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:22:39.031050 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:39.031689 11046 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:39.031838 11046 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:39.032711 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:44.033393 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:44.034443 11046 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:44.034685 11046 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:44.035321 11051 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:49.035828 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:49.036345 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:49.036553 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:49.037240 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:49.636526 11047 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:22:54.038525 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:22:54.039108 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:54.039549 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:54.039691 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:54.040293 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:22:59.040702 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:22:59.041216 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:22:59.041524 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:22:59.041749 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:22:59.042407 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:00.639822 11049 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:23:04.042881 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:04.043432 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:04.043550 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:04.044637 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> W0719 12:23:09.044903 11051 master.cpp:5541] Authentication timed out
>>>> W0719 12:23:09.045856 11049 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:09.046164 11049 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:09.046769 11049 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:11.640506 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:23:14.047653 11048 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:14.048249 11048 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:14.048429 11048 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:14.049170 11048 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:19.049494 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:19.050199 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:19.050398 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:19.051105 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:22.639598 11046 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:23:24.051370 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:24.052043 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:24.052255 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:24.053045 11045 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:29.053254 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:23:29.053788 11044 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:29.054085 11044 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:29.054247 11044 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:29.054867 11044 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:33.639268 11048 http.cpp:312] HTTP GET for /master/state
>>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>>> Safari/537.36'
>>>> I0719 12:23:34.055438 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> I0719 12:23:34.055974 11047 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:34.056213 11047 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:34.056349 11047 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:34.056926 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:39.057729 11045 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0719 12:23:39.058291 11045 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>> I0719 12:23:39.058435 11045 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0719 12:23:39.059099 11047 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0719 12:23:44.059839 11051 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10 <http://10.5.7.5:5051>
>>>>
>>> ...
>>
>> [Message clipped]
>
>
>

Re: Possible authentication bug

Posted by Douglas Nelson <it...@gmail.com>.

I tried setting --authentication_backoff_factor=9secs to experiment with a
different time. The mesos agent doesn't start at all when I set that flag?
I'm running mesos 0.28.2.

On Wed, Jul 20, 2016 at 9:12 PM, Adam Bordelon <ad...@mesosphere.io> wrote:

> This does indeed look identical to the logs from previous occurrences of
> https://issues.apache.org/jira/browse/MESOS-2043
> This can happen when your master's message queue gets clogged (check
> /metrics/snapshot). The scheduler/agent begins authentication and starts
> its timeout counter. Eventually, the master gets the message out of its
> queue and asks the scheduler/agent for the next step in authentication. If
> more than 5 seconds have passed since the scheduler/agent started its
> timeout counter, then the agent will restart authentication, causing the
> master to discard the previous authentication and start the process all
> over again.
> What we did in MESOS-2043 is add an increasing backoff to the
> authentication timeout, so that we'll eventually wait long enough for an
> authentication messages to make it through the queue a few times.
>
> On Wed, Jul 20, 2016 at 4:17 PM, Douglas Nelson <it...@gmail.com>
> wrote:
>
>> I discovered something else with this issue today. Authentication only
>> fails every other time mesos attempts to register an agent node with the
>> master node. I've tried completely clearing out zookeeper and starting from
>> scratch with the same result. I deploy an application using marathon, kill
>> the agent, wait for it to unregister with the master, then start the agent
>> again. Repeating that process, every other time it's able to register. When
>> it isn't able to register I have to restart the master node before
>> registration completes. Could it be related to the app deployed by
>> marathon? Has anyone else seen this behavior?
>>
>> If any other information is needed, let me know. I'm out of ideas about
>> the cause.
>>
>> On Tue, Jul 19, 2016 at 12:47 PM, Douglas Nelson <it...@gmail.com>
>> wrote:
>>
>>> MASTER:
>>>
>>> Log file created at: 2016/07/19 12:13:55
>>> Running on machine: dev1
>>> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
>>> I0719 12:13:55.437517 11025 logging.cpp:188] INFO level logging started!
>>> I0719 12:13:55.438292 11025 main.cpp:237] Build: 2016-07-08 16:57:43 by
>>> root
>>> I0719 12:13:55.438343 11025 main.cpp:239] Version: 0.28.2
>>> I0719 12:13:55.438967 11025 main.cpp:260] Using 'HierarchicalDRF'
>>> allocator
>>> I0719 12:13:55.449280 11025 leveldb.cpp:174] Opened db in 9.870872ms
>>> I0719 12:13:55.453482 11025 leveldb.cpp:181] Compacted db in 4.087025ms
>>> I0719 12:13:55.453719 11025 leveldb.cpp:196] Created db iterator in
>>> 132186ns
>>> I0719 12:13:55.453820 11025 leveldb.cpp:202] Seeked to beginning of db
>>> in 35181ns
>>> I0719 12:13:55.454460 11025 leveldb.cpp:271] Iterated through 3 keys in
>>> the db in 592652ns
>>> I0719 12:13:55.454795 11025 replica.cpp:779] Replica recovered with log
>>> positions 129 -> 130 with 0 holes and 0 unlearned
>>> I0719 12:13:55.461314 11049 log.cpp:236] Attempting to join replica to
>>> ZooKeeper group
>>> I0719 12:13:55.465665 11025 main.cpp:471] Starting Mesos master
>>> I0719 12:13:55.478101 11048 recover.cpp:447] Starting replica recovery
>>> I0719 12:13:55.482789 11025 master.cpp:375] Master
>>> c3c11abc-2f26-479f-8965-db34be5aa567 (dev1.something.com) started on
>>> 10.5.7.4:5050
>>> I0719 12:13:55.482998 11025 master.cpp:377] Flags at startup:
>>> --allocation_interval="1secs" --allocator="HierarchicalDRF"
>>> --authenticate="true" --authenticate_http="true"
>>> --authenticate_slaves="true" --authenticators="crammd5"
>>> --authorizers="local" --credentials="/etc/mesos/auth/credentials"
>>> --framework_sorter="drf" --help="false" --hostname_lookup="true"
>>> --http_authenticators="basic" --initialize_driver_logging="true"
>>> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
>>> --logging_level="INFO" --max_completed_frameworks="50"
>>> --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
>>> --port="5050" --quiet="false" --quorum="1"
>>> --recovery_slave_removal_limit="100%" --registry="replicated_log"
>>> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
>>> --registry_strict="false" --root_submissions="true"
>>> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
>>> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
>>> --work_dir="/var/lib/mesos" --zk="zk://10.5.7.4:2181/mesos"
>>> --zk_session_timeout="10secs"
>>> I0719 12:13:55.483695 11025 master.cpp:422] Master only allowing
>>> authenticated frameworks to register
>>> I0719 12:13:55.483753 11025 master.cpp:427] Master only allowing
>>> authenticated slaves to register
>>> I0719 12:13:55.483783 11025 credentials.hpp:35] Loading credentials for
>>> authentication from '/etc/mesos/auth/credentials'
>>> W0719 12:13:55.484066 11025 credentials.hpp:50] Permissions on
>>> credentials file '/etc/mesos/auth/credentials' are too open. It is
>>> recommended that your credentials file is NOT accessible by others.
>>> I0719 12:13:55.484253 11048 recover.cpp:473] Replica is in VOTING status
>>> I0719 12:13:55.484794 11048 recover.cpp:462] Recover process terminated
>>> I0719 12:13:55.494889 11025 master.cpp:467] Using default 'crammd5'
>>> authenticator
>>> I0719 12:13:55.495386 11025 authenticator.cpp:518] Initializing server
>>> SASL
>>> I0719 12:13:55.498042 11025 master.cpp:536] Using default 'basic' HTTP
>>> authenticator
>>> I0719 12:13:55.511752 11025 master.cpp:1650] Successfully attached file
>>> '/var/log/mesos/mesos-master.INFO'
>>> I0719 12:13:55.512787 11046 contender.cpp:147] Joining the ZK group
>>> I0719 12:13:55.603083 11046 group.cpp:349] Group process (group(1)@
>>> 10.5.7.4:5050) connected to ZooKeeper
>>> I0719 12:13:55.603395 11046 group.cpp:831] Syncing group operations:
>>> queue size (joins, cancels, datas) = (0, 0, 0)
>>> I0719 12:13:55.603469 11046 group.cpp:427] Trying to create path
>>> '/mesos/log_replicas' in ZooKeeper
>>> I0719 12:13:55.610569 11051 group.cpp:349] Group process (group(4)@
>>> 10.5.7.4:5050) connected to ZooKeeper
>>> I0719 12:13:55.610783 11051 group.cpp:831] Syncing group operations:
>>> queue size (joins, cancels, datas) = (0, 0, 0)
>>> I0719 12:13:55.610844 11051 group.cpp:427] Trying to create path
>>> '/mesos' in ZooKeeper
>>> I0719 12:13:55.610780 11044 group.cpp:349] Group process (group(3)@
>>> 10.5.7.4:5050) connected to ZooKeeper
>>> I0719 12:13:55.611119 11044 group.cpp:831] Syncing group operations:
>>> queue size (joins, cancels, datas) = (1, 0, 0)
>>> I0719 12:13:55.611163 11044 group.cpp:427] Trying to create path
>>> '/mesos' in ZooKeeper
>>> I0719 12:13:55.611829 11049 group.cpp:349] Group process (group(2)@
>>> 10.5.7.4:5050) connected to ZooKeeper
>>> I0719 12:13:55.611938 11049 group.cpp:831] Syncing group operations:
>>> queue size (joins, cancels, datas) = (1, 0, 0)
>>> I0719 12:13:55.611970 11049 group.cpp:427] Trying to create path
>>> '/mesos/log_replicas' in ZooKeeper
>>> I0719 12:13:55.652545 11048 contender.cpp:263] New candidate (id='2')
>>> has entered the contest for leadership
>>> I0719 12:13:55.656810 11046 network.hpp:413] ZooKeeper group memberships
>>> changed
>>> I0719 12:13:55.657207 11044 group.cpp:700] Trying to get
>>> '/mesos/log_replicas/0000000001' in ZooKeeper
>>> I0719 12:13:55.659225 11049 detector.cpp:152] Detected a new leader:
>>> (id='2')
>>> I0719 12:13:55.660096 11051 group.cpp:700] Trying to get
>>> '/mesos/json.info_0000000002' in ZooKeeper
>>> I0719 12:13:55.663472 11050 network.hpp:461] ZooKeeper group PIDs: {
>>> log-replica(1)@10.5.7.4:5050 }
>>> I0719 12:13:55.664608 11046 detector.cpp:479] A new leading master (UPID=
>>> master@10.5.7.4:5050) is detected
>>> I0719 12:13:55.665132 11049 master.cpp:1711] The newly elected leader is
>>> master@10.5.7.4:5050 with id c3c11abc-2f26-479f-8965-db34be5aa567
>>> I0719 12:13:55.665240 11049 master.cpp:1724] Elected as the leading
>>> master!
>>> I0719 12:13:55.665334 11049 master.cpp:1469] Recovering from registrar
>>> I0719 12:13:55.665786 11044 registrar.cpp:307] Recovering registrar
>>> I0719 12:13:55.668251 11049 log.cpp:659] Attempting to start the writer
>>> I0719 12:13:55.674710 11045 replica.cpp:493] Replica received implicit
>>> promise request from (6)@10.5.7.4:5050 with proposal 30
>>> I0719 12:13:55.676144 11045 leveldb.cpp:304] Persisting metadata (8
>>> bytes) to leveldb took 1.29057ms
>>> I0719 12:13:55.676283 11045 replica.cpp:342] Persisted promised to 30
>>> I0719 12:13:55.678892 11044 coordinator.cpp:238] Coordinator attempting
>>> to fill missing positions
>>> I0719 12:13:55.679973 11047 log.cpp:675] Writer started with ending
>>> position 130
>>> I0719 12:13:55.684237 11051 leveldb.cpp:436] Reading position from
>>> leveldb took 180882ns
>>> I0719 12:13:55.684692 11051 leveldb.cpp:436] Reading position from
>>> leveldb took 86170ns
>>> I0719 12:13:55.689906 11045 registrar.cpp:340] Successfully fetched the
>>> registry (320B) in 23.838976ms
>>> I0719 12:13:55.690568 11045 registrar.cpp:439] Applied 1 operations in
>>> 253986ns; attempting to update the 'registry'
>>> I0719 12:13:55.695715 11051 log.cpp:683] Attempting to append 359 bytes
>>> to the log
>>> I0719 12:13:55.696043 11045 coordinator.cpp:348] Coordinator attempting
>>> to write APPEND action at position 131
>>> I0719 12:13:55.698156 11045 replica.cpp:537] Replica received write
>>> request for position 131 from (7)@10.5.7.4:5050
>>> I0719 12:13:55.699312 11045 leveldb.cpp:341] Persisting action (379
>>> bytes) to leveldb took 918900ns
>>> I0719 12:13:55.699405 11045 replica.cpp:712] Persisted action at 131
>>> I0719 12:13:55.700897 11048 replica.cpp:691] Replica received learned
>>> notice for position 131 from @0.0.0.0:0
>>> I0719 12:13:55.701294 11048 leveldb.cpp:341] Persisting action (381
>>> bytes) to leveldb took 305226ns
>>> I0719 12:13:55.701371 11048 replica.cpp:712] Persisted action at 131
>>> I0719 12:13:55.701427 11048 replica.cpp:697] Replica learned APPEND
>>> action at position 131
>>> I0719 12:13:55.705004 11051 registrar.cpp:484] Successfully updated the
>>> 'registry' in 14.150144ms
>>> I0719 12:13:55.705585 11051 registrar.cpp:370] Successfully recovered
>>> registrar
>>> I0719 12:13:55.705776 11047 log.cpp:702] Attempting to truncate the log
>>> to 131
>>> I0719 12:13:55.706045 11046 coordinator.cpp:348] Coordinator attempting
>>> to write TRUNCATE action at position 132
>>> I0719 12:13:55.707181 11044 replica.cpp:537] Replica received write
>>> request for position 132 from (8)@10.5.7.4:5050
>>> I0719 12:13:55.707342 11046 master.cpp:1521] Recovered 1 slaves from the
>>> Registry (320B) ; allowing 10mins for slaves to re-register
>>> I0719 12:13:55.708180 11044 leveldb.cpp:341] Persisting action (18
>>> bytes) to leveldb took 889855ns
>>> I0719 12:13:55.708283 11044 replica.cpp:712] Persisted action at 132
>>> I0719 12:13:55.709445 11051 replica.cpp:691] Replica received learned
>>> notice for position 132 from @0.0.0.0:0
>>> I0719 12:13:55.709847 11051 leveldb.cpp:341] Persisting action (20
>>> bytes) to leveldb took 300975ns
>>> I0719 12:13:55.709995 11051 leveldb.cpp:399] Deleting ~2 keys from
>>> leveldb took 69236ns
>>> I0719 12:13:55.710054 11051 replica.cpp:712] Persisted action at 132
>>> I0719 12:13:55.710115 11051 replica.cpp:697] Replica learned TRUNCATE
>>> action at position 132
>>> I0719 12:14:05.059528 11047 master.cpp:5495] Authenticating
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:05.062243 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:14:05.098544 11049 authenticator.cpp:203] Received SASL
>>> authentication start
>>> I0719 12:14:05.098788 11049 authenticator.cpp:325] Authentication
>>> requires more steps
>>> I0719 12:14:05.107225 11044 authenticator.cpp:231] Received SASL
>>> authentication step
>>> I0719 12:14:05.107561 11044 authenticator.cpp:317] Authentication success
>>> I0719 12:14:05.107964 11044 master.cpp:5525] Successfully authenticated
>>> principal 'dev' at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:05.110188 11046 master.cpp:2231] Received SUBSCRIBE call for
>>> framework 'marathon' at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:05.110599 11046 master.cpp:2302] Subscribing framework
>>> marathon with checkpointing enabled and capabilities [  ]
>>> I0719 12:14:05.112211 11051 hierarchical.cpp:265] Added framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:14:07.229689 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:14:07.250246 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:14:07.281565 11050 authenticator.cpp:203] Received SASL
>>> authentication start
>>> I0719 12:14:07.281793 11050 authenticator.cpp:325] Authentication
>>> requires more steps
>>> I0719 12:14:07.301153 11050 authenticator.cpp:231] Received SASL
>>> authentication step
>>> I0719 12:14:07.301486 11050 authenticator.cpp:317] Authentication success
>>> I0719 12:14:07.302227 11051 master.cpp:5525] Successfully authenticated
>>> principal 'dev' at slave(1)@10.5.7.5:5051
>>> I0719 12:14:07.307806 11051 master.cpp:4439] Re-registering slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com)
>>> I0719 12:14:07.310359 11044 registrar.cpp:439] Applied 1 operations in
>>> 176631ns; attempting to update the 'registry'
>>> I0719 12:14:07.316319 11049 log.cpp:683] Attempting to append 359 bytes
>>> to the log
>>> I0719 12:14:07.316714 11046 coordinator.cpp:348] Coordinator attempting
>>> to write APPEND action at position 133
>>> I0719 12:14:07.318164 11046 replica.cpp:537] Replica received write
>>> request for position 133 from (11)@10.5.7.4:5050
>>> I0719 12:14:07.318792 11046 leveldb.cpp:341] Persisting action (379
>>> bytes) to leveldb took 519351ns
>>> I0719 12:14:07.318903 11046 replica.cpp:712] Persisted action at 133
>>> I0719 12:14:07.320116 11048 replica.cpp:691] Replica received learned
>>> notice for position 133 from @0.0.0.0:0
>>> I0719 12:14:07.320976 11048 leveldb.cpp:341] Persisting action (381
>>> bytes) to leveldb took 785461ns
>>> I0719 12:14:07.321053 11048 replica.cpp:712] Persisted action at 133
>>> I0719 12:14:07.321106 11048 replica.cpp:697] Replica learned APPEND
>>> action at position 133
>>> I0719 12:14:07.323272 11046 registrar.cpp:484] Successfully updated the
>>> 'registry' in 12.685056ms
>>> I0719 12:14:07.323559 11048 log.cpp:702] Attempting to truncate the log
>>> to 133
>>> I0719 12:14:07.323832 11049 coordinator.cpp:348] Coordinator attempting
>>> to write TRUNCATE action at position 134
>>> I0719 12:14:07.324877 11049 replica.cpp:537] Replica received write
>>> request for position 134 from (12)@10.5.7.4:5050
>>> I0719 12:14:07.325728 11049 leveldb.cpp:341] Persisting action (18
>>> bytes) to leveldb took 756749ns
>>> I0719 12:14:07.325804 11049 replica.cpp:712] Persisted action at 134
>>> W0719 12:14:07.326323 11047 master.cpp:6170] Possibly orphaned completed
>>> task webapp.42f99194-4dda-11e6-8041-02421568c935 of framework
>>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001 that ran on slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com)
>>> I0719 12:14:07.326839 11049 replica.cpp:691] Replica received learned
>>> notice for position 134 from @0.0.0.0:0
>>> I0719 12:14:07.327742 11049 leveldb.cpp:341] Persisting action (20
>>> bytes) to leveldb took 717423ns
>>> I0719 12:14:07.327713 11047 master.cpp:4521] Re-registered slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com) with cpus(*):4; mem(*):2929; disk(*):242457;
>>> ports(*):[31000-32000]
>>> I0719 12:14:07.328063 11047 master.cpp:4556] Sending updated
>>> checkpointed resources  to slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at
>>> slave(1)@10.5.7.5:5051 (dev2.something.com)
>>> I0719 12:14:07.328157 11049 leveldb.cpp:399] Deleting ~2 keys from
>>> leveldb took 188898ns
>>> I0719 12:14:07.328233 11049 replica.cpp:712] Persisted action at 134
>>> I0719 12:14:07.328300 11049 replica.cpp:697] Replica learned TRUNCATE
>>> action at position 134
>>> I0719 12:14:07.329198 11046 hierarchical.cpp:473] Added slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) with
>>> cpus(*):4; mem(*):2929; disk(*):242457; ports(*):[31000-32000] (allocated: )
>>> I0719 12:14:07.330371 11044 master.cpp:4618] Received update of slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com) with total oversubscribed resources
>>> I0719 12:14:07.332751 11046 hierarchical.cpp:531] Slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) updated
>>> with oversubscribed resources  (total: cpus(*):4; mem(*):2929;
>>> disk(*):242457; ports(*):[31000-32000], allocated: cpus(*):4; mem(*):2929;
>>> disk(*):242457; ports(*):[31000-32000])
>>> I0719 12:14:07.332794 11047 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:07.432186 11045 master.cpp:3641] Processing DECLINE call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O0 ] for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:09.503839 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:14:09.877415 11044 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:09.877856 11044 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:14:09.880434 11051 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:09.886831 11048 master.cpp:3641] Processing DECLINE call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O1 ] for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:14.894800 11045 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:14.895149 11051 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:14:14.897150 11045 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:14.903162 11044 master.cpp:3641] Processing DECLINE call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O2 ] for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:19.714318 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:14:19.914870 11049 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:19.915210 11047 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:14:19.917094 11049 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:19.923250 11051 master.cpp:3641] Processing DECLINE call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O3 ] for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:20.019886 11050 master.cpp:4997] Performing implicit task
>>> state reconciliation for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:29.515525 11048 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:29.515820 11048 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:14:29.517138 11048 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:29.766815 11044 master.cpp:3104] Processing ACCEPT call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O4 ] on slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com) for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:29.771697 11049 master.hpp:177] Adding task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>>> mem(*):32; ports(*):[31990-31990] on slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>>> I0719 12:14:29.772017 11049 master.cpp:3589] Launching task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 with
>>> resources cpus(*):0.1; mem(*):32; ports(*):[31990-31990] on slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com)
>>> I0719 12:14:30.589148 11045 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:30.595021 11046 master.cpp:3641] Processing DECLINE call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O5 ] for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:14:30.610872 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:14:30.803971 11048 master.cpp:4763] Status update TASK_RUNNING
>>> (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 from slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com)
>>> I0719 12:14:30.804188 11048 master.cpp:4811] Forwarding status update
>>> TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:14:30.804988 11048 master.cpp:6421] Updating the state of task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_RUNNING,
>>> status update state: TASK_RUNNING)
>>> I0719 12:14:30.868975 11046 master.cpp:3918] Processing ACKNOWLEDGE call
>>> 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f for task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 on slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>>> I0719 12:14:40.864478 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:14:51.114728 11050 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:01.365391 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:08.869536 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51290 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:18.887311 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:28.904626 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:38.918462 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:48.932732 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:15:58.946310 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:16:08.961182 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:16:18.976491 11049 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:16:29.148555 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:16:30.951886 11046 master.cpp:5324] Sending 1 offers to
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:16:30.958127 11047 master.cpp:3641] Processing DECLINE call for
>>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O6 ] for framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:16:39.398538 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:16:49.648846 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:16:59.899797 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:17:07.350857 11047 master.cpp:242] Shutting down slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 due to health check timeout
>>> W0719 12:17:07.351191 11047 master.cpp:4930] Shutting down slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com) with message 'health check timed out'
>>> I0719 12:17:07.351920 11047 master.cpp:6201] Removing slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com): health check timed out
>>> I0719 12:17:07.352468 11047 master.cpp:6421] Updating the state of task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_LOST, status
>>> update state: TASK_LOST)
>>> I0719 12:17:07.352746 11045 hierarchical.cpp:505] Removed slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>>> I0719 12:17:07.353837 11047 master.cpp:6487] Removing task
>>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>>> mem(*):32; ports(*):[31990-31990] of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 on slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>>> dev2.something.com)
>>> I0719 12:17:07.356307 11046 registrar.cpp:439] Applied 1 operations in
>>> 164323ns; attempting to update the 'registry'
>>> I0719 12:17:07.360155 11045 log.cpp:683] Attempting to append 184 bytes
>>> to the log
>>> I0719 12:17:07.360399 11050 coordinator.cpp:348] Coordinator attempting
>>> to write APPEND action at position 135
>>> I0719 12:17:07.361938 11051 replica.cpp:537] Replica received write
>>> request for position 135 from (40)@10.5.7.4:5050
>>> I0719 12:17:07.362778 11051 leveldb.cpp:341] Persisting action (204
>>> bytes) to leveldb took 728456ns
>>> I0719 12:17:07.362862 11051 replica.cpp:712] Persisted action at 135
>>> I0719 12:17:07.363806 11051 replica.cpp:691] Replica received learned
>>> notice for position 135 from @0.0.0.0:0
>>> I0719 12:17:07.364334 11051 leveldb.cpp:341] Persisting action (206
>>> bytes) to leveldb took 430978ns
>>> I0719 12:17:07.364414 11051 replica.cpp:712] Persisted action at 135
>>> I0719 12:17:07.364471 11051 replica.cpp:697] Replica learned APPEND
>>> action at position 135
>>> I0719 12:17:07.366155 11047 registrar.cpp:484] Successfully updated the
>>> 'registry' in 9.689088ms
>>> I0719 12:17:07.366540 11051 log.cpp:702] Attempting to truncate the log
>>> to 135
>>> I0719 12:17:07.366657 11046 master.cpp:6320] Removed slave
>>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com): health
>>> check timed out
>>> I0719 12:17:07.366739 11047 coordinator.cpp:348] Coordinator attempting
>>> to write TRUNCATE action at position 136
>>> I0719 12:17:07.366789 11046 master.cpp:4806] Sending status update
>>> TASK_LOST for task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 'Slave dev2.something.com
>>> removed: health check timed out'
>>> I0719 12:17:07.367236 11046 master.cpp:6343] Notifying framework
>>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 of lost
>>> slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>>> after recovering
>>> I0719 12:17:07.367993 11049 replica.cpp:537] Replica received write
>>> request for position 136 from (41)@10.5.7.4:5050
>>> I0719 12:17:07.368701 11049 leveldb.cpp:341] Persisting action (18
>>> bytes) to leveldb took 607673ns
>>> I0719 12:17:07.368780 11049 replica.cpp:712] Persisted action at 136
>>> I0719 12:17:07.369793 11048 replica.cpp:691] Replica received learned
>>> notice for position 136 from @0.0.0.0:0
>>> I0719 12:17:07.374290 11048 leveldb.cpp:341] Persisting action (20
>>> bytes) to leveldb took 4.333673ms
>>> I0719 12:17:07.374531 11048 leveldb.cpp:399] Deleting ~2 keys from
>>> leveldb took 142498ns
>>> I0719 12:17:07.374651 11048 replica.cpp:712] Persisted action at 136
>>> I0719 12:17:07.374763 11048 replica.cpp:697] Replica learned TRUNCATE
>>> action at position 136
>>> I0719 12:17:07.413800 11048 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:17:07.414114 11051 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:17:10.107131 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:17:12.435168 11051 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:17:12.435470 11051 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:17:17.455554 11049 master.cpp:3720] Processing REVIVE call for
>>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>>> I0719 12:17:17.456195 11044 hierarchical.cpp:988] Removed offer filters
>>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>>> I0719 12:17:20.620965 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:17:28.907829 11051 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:28.908550 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:17:30.634392 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:17:33.887810 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:17:33.888640 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:17:33.888864 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:33.889627 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:17:38.890091 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:17:38.890705 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:17:38.890908 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:38.891654 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:17:41.619175 11049 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:17:43.892462 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:17:43.893380 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:17:43.893582 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:43.894261 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:17:48.894907 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:17:48.895597 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:17:48.895742 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:48.896458 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:17:52.622318 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:17:53.897758 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:17:53.898336 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:17:53.898510 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:53.899430 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:17:58.900116 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:17:58.900677 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:17:58.901741 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:17:58.902381 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:03.621628 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:18:03.902536 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:03.903519 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:03.903714 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:03.904594 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:08.904714 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:18:08.905208 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:08.905773 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:08.905913 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:08.906513 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:13.907012 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:18:13.907600 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:13.907958 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:13.908123 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:13.908871 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:14.624475 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:18:18.909356 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:18.910174 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:18.910326 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:18.911255 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:23.911778 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:18:23.912201 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:23.912739 11046 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:23.913128 11046 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:23.913684 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:25.624198 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:18:28.914320 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:28.914964 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:28.915148 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:28.915825 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:33.916498 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:33.917145 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:33.917282 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:33.918520 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:36.625072 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:18:38.918712 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:38.919631 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:38.920052 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:38.920876 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:43.921413 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:43.922125 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:43.922269 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:43.923209 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:47.627984 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:18:48.923180 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:48.923815 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:48.924010 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:48.924589 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:53.925963 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:53.926895 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:53.927124 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:53.927942 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:18:58.626113 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:18:58.927940 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:18:58.928613 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:18:58.928766 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:18:58.930127 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:03.930527 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:03.931396 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:03.931637 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:03.932543 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:08.932302 11048 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:19:08.932852 11048 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:08.933034 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:08.933212 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:08.933835 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:09.627965 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:19:13.934926 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:13.935514 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:13.935757 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:13.936638 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:18.936506 11048 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:19:18.937036 11048 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:18.937420 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:18.937613 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:18.938341 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:20.624949 11049 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:19:23.938432 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:23.939040 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:23.939214 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:23.940078 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:28.941543 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:28.942198 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:28.942435 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:28.943140 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:31.627547 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:19:33.943943 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:19:33.944599 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:33.944835 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:33.944988 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:33.945693 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:38.945986 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:19:38.946478 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:38.946812 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:38.946949 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:38.947438 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:42.628363 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:19:43.948642 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:43.949349 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:43.949550 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:43.950319 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:48.951385 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:48.951884 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:48.952000 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:48.952672 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:53.628947 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:19:53.954244 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:53.954866 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:53.955090 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:53.956058 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:19:58.956197 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:19:58.956797 11046 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:19:58.957006 11046 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:19:58.957918 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:03.958680 11048 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:03.959282 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:03.959450 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:03.960024 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:04.630003 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:20:08.961760 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:20:08.962337 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:08.962745 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:08.962887 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:08.963418 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:13.964138 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:13.964747 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:13.965031 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:13.965772 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:15.630905 11051 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:20:18.966778 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:18.967433 11051 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:18.967687 11051 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:18.968503 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:23.969029 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:23.969832 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:23.969976 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:23.970935 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:26.628504 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:20:28.971153 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:20:28.971578 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:28.971874 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:28.972044 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:28.972594 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:33.973855 11051 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:20:33.974304 11051 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:33.974576 11051 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:33.974689 11051 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:33.975183 11051 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:37.629881 11050 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:20:38.976158 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:38.977018 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:38.977200 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:38.978510 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:43.977927 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:43.978523 11046 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:43.978688 11046 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:43.979673 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:48.631947 11044 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:20:48.980355 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:48.981098 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:48.981258 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:48.982522 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:53.983012 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:53.983757 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:53.983986 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:53.984787 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:58.985410 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:20:58.985873 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:20:58.986018 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:20:58.986199 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:20:58.986820 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:20:59.630745 11049 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:21:03.987254 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:03.987884 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:03.988030 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:03.989013 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:08.989428 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:21:08.990020 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:08.990285 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:08.990468 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:08.990981 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:10.633591 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:21:13.992058 11051 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:13.992918 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:13.993191 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:13.994236 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:18.994022 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:21:18.994477 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:18.994962 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:18.995141 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:18.995689 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:21.647516 11050 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:21:23.996325 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:23.997014 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:23.997248 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:23.998145 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:28.998471 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:28.999652 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:28.999872 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:29.000628 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:32.635963 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:21:34.000746 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:34.001461 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:34.001718 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:34.002413 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:39.003231 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:39.003926 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:39.004137 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:39.004909 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:43.639329 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:21:44.005383 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:44.005898 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:44.006666 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:44.007246 11046 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:49.007254 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:49.007946 11051 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:49.008180 11051 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:49.009217 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:54.009101 11046 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:54.009840 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:54.010089 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:54.010777 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:21:54.634322 11050 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:21:59.011060 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:21:59.011662 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:21:59.011852 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:21:59.012444 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:04.013264 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:22:04.013864 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:04.014137 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:04.014339 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:04.015051 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:05.633569 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:22:09.015625 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:09.016332 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:09.016549 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:09.017400 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:14.018102 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:14.018810 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:14.018975 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:14.020356 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:16.638113 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:22:19.020917 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:22:19.021430 11049 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:19.021756 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:19.021895 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:19.022616 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:24.023571 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:24.024431 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:24.024754 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:24.026060 11050 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:27.643009 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:22:29.026162 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:29.026885 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:29.027034 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:29.028298 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:34.028923 11050 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:34.029616 11050 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:34.029865 11050 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:34.030767 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:38.638548 11045 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:22:39.031050 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:39.031689 11046 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:39.031838 11046 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:39.032711 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:44.033393 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:44.034443 11046 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:44.034685 11046 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:44.035321 11051 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:49.035828 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:49.036345 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:49.036553 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:49.037240 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:49.636526 11047 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:22:54.038525 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:22:54.039108 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:54.039549 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:54.039691 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:54.040293 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:22:59.040702 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:22:59.041216 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:22:59.041524 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:22:59.041749 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:22:59.042407 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:00.639822 11049 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:23:04.042881 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:04.043432 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:04.043550 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:04.044637 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> W0719 12:23:09.044903 11051 master.cpp:5541] Authentication timed out
>>> W0719 12:23:09.045856 11049 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:09.046164 11049 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:09.046769 11049 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:11.640506 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:23:14.047653 11048 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:14.048249 11048 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:14.048429 11048 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:14.049170 11048 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:19.049494 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:19.050199 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:19.050398 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:19.051105 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:22.639598 11046 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:23:24.051370 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:24.052043 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:24.052255 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:24.053045 11045 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:29.053254 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:23:29.053788 11044 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:29.054085 11044 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:29.054247 11044 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:29.054867 11044 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:33.639268 11048 http.cpp:312] HTTP GET for /master/state
>>> from 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1;
>>> WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103
>>> Safari/537.36'
>>> I0719 12:23:34.055438 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> I0719 12:23:34.055974 11047 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:34.056213 11047 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:34.056349 11047 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:34.056926 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:39.057729 11045 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0719 12:23:39.058291 11045 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>> I0719 12:23:39.058435 11045 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0719 12:23:39.059099 11047 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0719 12:23:44.059839 11051 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10 <http://10.5.7.5:5051>
>>>
>> ...
>
> [Message clipped]

Re: Possible authentication bug

Posted by Adam Bordelon <ad...@mesosphere.io>.

This does indeed look identical to the logs from previous occurrences of
https://issues.apache.org/jira/browse/MESOS-2043
This can happen when your master's message queue gets clogged (check
/metrics/snapshot). The scheduler/agent begins authentication and starts
its timeout counter. Eventually, the master gets the message out of its
queue and asks the scheduler/agent for the next step in authentication. If
more than 5 seconds have passed since the scheduler/agent started its
timeout counter, then the agent will restart authentication, causing the
master to discard the previous authentication and start the process all
over again.
What we did in MESOS-2043 is add an increasing backoff to the
authentication timeout, so that we'll eventually wait long enough for an
authentication messages to make it through the queue a few times.

On Wed, Jul 20, 2016 at 4:17 PM, Douglas Nelson <it...@gmail.com> wrote:

> I discovered something else with this issue today. Authentication only
> fails every other time mesos attempts to register an agent node with the
> master node. I've tried completely clearing out zookeeper and starting from
> scratch with the same result. I deploy an application using marathon, kill
> the agent, wait for it to unregister with the master, then start the agent
> again. Repeating that process, every other time it's able to register. When
> it isn't able to register I have to restart the master node before
> registration completes. Could it be related to the app deployed by
> marathon? Has anyone else seen this behavior?
>
> If any other information is needed, let me know. I'm out of ideas about
> the cause.
>
> On Tue, Jul 19, 2016 at 12:47 PM, Douglas Nelson <it...@gmail.com>
> wrote:
>
>> MASTER:
>>
>> Log file created at: 2016/07/19 12:13:55
>> Running on machine: dev1
>> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
>> I0719 12:13:55.437517 11025 logging.cpp:188] INFO level logging started!
>> I0719 12:13:55.438292 11025 main.cpp:237] Build: 2016-07-08 16:57:43 by
>> root
>> I0719 12:13:55.438343 11025 main.cpp:239] Version: 0.28.2
>> I0719 12:13:55.438967 11025 main.cpp:260] Using 'HierarchicalDRF'
>> allocator
>> I0719 12:13:55.449280 11025 leveldb.cpp:174] Opened db in 9.870872ms
>> I0719 12:13:55.453482 11025 leveldb.cpp:181] Compacted db in 4.087025ms
>> I0719 12:13:55.453719 11025 leveldb.cpp:196] Created db iterator in
>> 132186ns
>> I0719 12:13:55.453820 11025 leveldb.cpp:202] Seeked to beginning of db in
>> 35181ns
>> I0719 12:13:55.454460 11025 leveldb.cpp:271] Iterated through 3 keys in
>> the db in 592652ns
>> I0719 12:13:55.454795 11025 replica.cpp:779] Replica recovered with log
>> positions 129 -> 130 with 0 holes and 0 unlearned
>> I0719 12:13:55.461314 11049 log.cpp:236] Attempting to join replica to
>> ZooKeeper group
>> I0719 12:13:55.465665 11025 main.cpp:471] Starting Mesos master
>> I0719 12:13:55.478101 11048 recover.cpp:447] Starting replica recovery
>> I0719 12:13:55.482789 11025 master.cpp:375] Master
>> c3c11abc-2f26-479f-8965-db34be5aa567 (dev1.something.com) started on
>> 10.5.7.4:5050
>> I0719 12:13:55.482998 11025 master.cpp:377] Flags at startup:
>> --allocation_interval="1secs" --allocator="HierarchicalDRF"
>> --authenticate="true" --authenticate_http="true"
>> --authenticate_slaves="true" --authenticators="crammd5"
>> --authorizers="local" --credentials="/etc/mesos/auth/credentials"
>> --framework_sorter="drf" --help="false" --hostname_lookup="true"
>> --http_authenticators="basic" --initialize_driver_logging="true"
>> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
>> --logging_level="INFO" --max_completed_frameworks="50"
>> --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
>> --port="5050" --quiet="false" --quorum="1"
>> --recovery_slave_removal_limit="100%" --registry="replicated_log"
>> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
>> --registry_strict="false" --root_submissions="true"
>> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
>> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
>> --work_dir="/var/lib/mesos" --zk="zk://10.5.7.4:2181/mesos"
>> --zk_session_timeout="10secs"
>> I0719 12:13:55.483695 11025 master.cpp:422] Master only allowing
>> authenticated frameworks to register
>> I0719 12:13:55.483753 11025 master.cpp:427] Master only allowing
>> authenticated slaves to register
>> I0719 12:13:55.483783 11025 credentials.hpp:35] Loading credentials for
>> authentication from '/etc/mesos/auth/credentials'
>> W0719 12:13:55.484066 11025 credentials.hpp:50] Permissions on
>> credentials file '/etc/mesos/auth/credentials' are too open. It is
>> recommended that your credentials file is NOT accessible by others.
>> I0719 12:13:55.484253 11048 recover.cpp:473] Replica is in VOTING status
>> I0719 12:13:55.484794 11048 recover.cpp:462] Recover process terminated
>> I0719 12:13:55.494889 11025 master.cpp:467] Using default 'crammd5'
>> authenticator
>> I0719 12:13:55.495386 11025 authenticator.cpp:518] Initializing server
>> SASL
>> I0719 12:13:55.498042 11025 master.cpp:536] Using default 'basic' HTTP
>> authenticator
>> I0719 12:13:55.511752 11025 master.cpp:1650] Successfully attached file
>> '/var/log/mesos/mesos-master.INFO'
>> I0719 12:13:55.512787 11046 contender.cpp:147] Joining the ZK group
>> I0719 12:13:55.603083 11046 group.cpp:349] Group process (group(1)@
>> 10.5.7.4:5050) connected to ZooKeeper
>> I0719 12:13:55.603395 11046 group.cpp:831] Syncing group operations:
>> queue size (joins, cancels, datas) = (0, 0, 0)
>> I0719 12:13:55.603469 11046 group.cpp:427] Trying to create path
>> '/mesos/log_replicas' in ZooKeeper
>> I0719 12:13:55.610569 11051 group.cpp:349] Group process (group(4)@
>> 10.5.7.4:5050) connected to ZooKeeper
>> I0719 12:13:55.610783 11051 group.cpp:831] Syncing group operations:
>> queue size (joins, cancels, datas) = (0, 0, 0)
>> I0719 12:13:55.610844 11051 group.cpp:427] Trying to create path '/mesos'
>> in ZooKeeper
>> I0719 12:13:55.610780 11044 group.cpp:349] Group process (group(3)@
>> 10.5.7.4:5050) connected to ZooKeeper
>> I0719 12:13:55.611119 11044 group.cpp:831] Syncing group operations:
>> queue size (joins, cancels, datas) = (1, 0, 0)
>> I0719 12:13:55.611163 11044 group.cpp:427] Trying to create path '/mesos'
>> in ZooKeeper
>> I0719 12:13:55.611829 11049 group.cpp:349] Group process (group(2)@
>> 10.5.7.4:5050) connected to ZooKeeper
>> I0719 12:13:55.611938 11049 group.cpp:831] Syncing group operations:
>> queue size (joins, cancels, datas) = (1, 0, 0)
>> I0719 12:13:55.611970 11049 group.cpp:427] Trying to create path
>> '/mesos/log_replicas' in ZooKeeper
>> I0719 12:13:55.652545 11048 contender.cpp:263] New candidate (id='2') has
>> entered the contest for leadership
>> I0719 12:13:55.656810 11046 network.hpp:413] ZooKeeper group memberships
>> changed
>> I0719 12:13:55.657207 11044 group.cpp:700] Trying to get
>> '/mesos/log_replicas/0000000001' in ZooKeeper
>> I0719 12:13:55.659225 11049 detector.cpp:152] Detected a new leader:
>> (id='2')
>> I0719 12:13:55.660096 11051 group.cpp:700] Trying to get
>> '/mesos/json.info_0000000002' in ZooKeeper
>> I0719 12:13:55.663472 11050 network.hpp:461] ZooKeeper group PIDs: {
>> log-replica(1)@10.5.7.4:5050 }
>> I0719 12:13:55.664608 11046 detector.cpp:479] A new leading master (UPID=
>> master@10.5.7.4:5050) is detected
>> I0719 12:13:55.665132 11049 master.cpp:1711] The newly elected leader is
>> master@10.5.7.4:5050 with id c3c11abc-2f26-479f-8965-db34be5aa567
>> I0719 12:13:55.665240 11049 master.cpp:1724] Elected as the leading
>> master!
>> I0719 12:13:55.665334 11049 master.cpp:1469] Recovering from registrar
>> I0719 12:13:55.665786 11044 registrar.cpp:307] Recovering registrar
>> I0719 12:13:55.668251 11049 log.cpp:659] Attempting to start the writer
>> I0719 12:13:55.674710 11045 replica.cpp:493] Replica received implicit
>> promise request from (6)@10.5.7.4:5050 with proposal 30
>> I0719 12:13:55.676144 11045 leveldb.cpp:304] Persisting metadata (8
>> bytes) to leveldb took 1.29057ms
>> I0719 12:13:55.676283 11045 replica.cpp:342] Persisted promised to 30
>> I0719 12:13:55.678892 11044 coordinator.cpp:238] Coordinator attempting
>> to fill missing positions
>> I0719 12:13:55.679973 11047 log.cpp:675] Writer started with ending
>> position 130
>> I0719 12:13:55.684237 11051 leveldb.cpp:436] Reading position from
>> leveldb took 180882ns
>> I0719 12:13:55.684692 11051 leveldb.cpp:436] Reading position from
>> leveldb took 86170ns
>> I0719 12:13:55.689906 11045 registrar.cpp:340] Successfully fetched the
>> registry (320B) in 23.838976ms
>> I0719 12:13:55.690568 11045 registrar.cpp:439] Applied 1 operations in
>> 253986ns; attempting to update the 'registry'
>> I0719 12:13:55.695715 11051 log.cpp:683] Attempting to append 359 bytes
>> to the log
>> I0719 12:13:55.696043 11045 coordinator.cpp:348] Coordinator attempting
>> to write APPEND action at position 131
>> I0719 12:13:55.698156 11045 replica.cpp:537] Replica received write
>> request for position 131 from (7)@10.5.7.4:5050
>> I0719 12:13:55.699312 11045 leveldb.cpp:341] Persisting action (379
>> bytes) to leveldb took 918900ns
>> I0719 12:13:55.699405 11045 replica.cpp:712] Persisted action at 131
>> I0719 12:13:55.700897 11048 replica.cpp:691] Replica received learned
>> notice for position 131 from @0.0.0.0:0
>> I0719 12:13:55.701294 11048 leveldb.cpp:341] Persisting action (381
>> bytes) to leveldb took 305226ns
>> I0719 12:13:55.701371 11048 replica.cpp:712] Persisted action at 131
>> I0719 12:13:55.701427 11048 replica.cpp:697] Replica learned APPEND
>> action at position 131
>> I0719 12:13:55.705004 11051 registrar.cpp:484] Successfully updated the
>> 'registry' in 14.150144ms
>> I0719 12:13:55.705585 11051 registrar.cpp:370] Successfully recovered
>> registrar
>> I0719 12:13:55.705776 11047 log.cpp:702] Attempting to truncate the log
>> to 131
>> I0719 12:13:55.706045 11046 coordinator.cpp:348] Coordinator attempting
>> to write TRUNCATE action at position 132
>> I0719 12:13:55.707181 11044 replica.cpp:537] Replica received write
>> request for position 132 from (8)@10.5.7.4:5050
>> I0719 12:13:55.707342 11046 master.cpp:1521] Recovered 1 slaves from the
>> Registry (320B) ; allowing 10mins for slaves to re-register
>> I0719 12:13:55.708180 11044 leveldb.cpp:341] Persisting action (18 bytes)
>> to leveldb took 889855ns
>> I0719 12:13:55.708283 11044 replica.cpp:712] Persisted action at 132
>> I0719 12:13:55.709445 11051 replica.cpp:691] Replica received learned
>> notice for position 132 from @0.0.0.0:0
>> I0719 12:13:55.709847 11051 leveldb.cpp:341] Persisting action (20 bytes)
>> to leveldb took 300975ns
>> I0719 12:13:55.709995 11051 leveldb.cpp:399] Deleting ~2 keys from
>> leveldb took 69236ns
>> I0719 12:13:55.710054 11051 replica.cpp:712] Persisted action at 132
>> I0719 12:13:55.710115 11051 replica.cpp:697] Replica learned TRUNCATE
>> action at position 132
>> I0719 12:14:05.059528 11047 master.cpp:5495] Authenticating
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:05.062243 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:14:05.098544 11049 authenticator.cpp:203] Received SASL
>> authentication start
>> I0719 12:14:05.098788 11049 authenticator.cpp:325] Authentication
>> requires more steps
>> I0719 12:14:05.107225 11044 authenticator.cpp:231] Received SASL
>> authentication step
>> I0719 12:14:05.107561 11044 authenticator.cpp:317] Authentication success
>> I0719 12:14:05.107964 11044 master.cpp:5525] Successfully authenticated
>> principal 'dev' at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:05.110188 11046 master.cpp:2231] Received SUBSCRIBE call for
>> framework 'marathon' at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:05.110599 11046 master.cpp:2302] Subscribing framework
>> marathon with checkpointing enabled and capabilities [  ]
>> I0719 12:14:05.112211 11051 hierarchical.cpp:265] Added framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:14:07.229689 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:14:07.250246 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:14:07.281565 11050 authenticator.cpp:203] Received SASL
>> authentication start
>> I0719 12:14:07.281793 11050 authenticator.cpp:325] Authentication
>> requires more steps
>> I0719 12:14:07.301153 11050 authenticator.cpp:231] Received SASL
>> authentication step
>> I0719 12:14:07.301486 11050 authenticator.cpp:317] Authentication success
>> I0719 12:14:07.302227 11051 master.cpp:5525] Successfully authenticated
>> principal 'dev' at slave(1)@10.5.7.5:5051
>> I0719 12:14:07.307806 11051 master.cpp:4439] Re-registering slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com)
>> I0719 12:14:07.310359 11044 registrar.cpp:439] Applied 1 operations in
>> 176631ns; attempting to update the 'registry'
>> I0719 12:14:07.316319 11049 log.cpp:683] Attempting to append 359 bytes
>> to the log
>> I0719 12:14:07.316714 11046 coordinator.cpp:348] Coordinator attempting
>> to write APPEND action at position 133
>> I0719 12:14:07.318164 11046 replica.cpp:537] Replica received write
>> request for position 133 from (11)@10.5.7.4:5050
>> I0719 12:14:07.318792 11046 leveldb.cpp:341] Persisting action (379
>> bytes) to leveldb took 519351ns
>> I0719 12:14:07.318903 11046 replica.cpp:712] Persisted action at 133
>> I0719 12:14:07.320116 11048 replica.cpp:691] Replica received learned
>> notice for position 133 from @0.0.0.0:0
>> I0719 12:14:07.320976 11048 leveldb.cpp:341] Persisting action (381
>> bytes) to leveldb took 785461ns
>> I0719 12:14:07.321053 11048 replica.cpp:712] Persisted action at 133
>> I0719 12:14:07.321106 11048 replica.cpp:697] Replica learned APPEND
>> action at position 133
>> I0719 12:14:07.323272 11046 registrar.cpp:484] Successfully updated the
>> 'registry' in 12.685056ms
>> I0719 12:14:07.323559 11048 log.cpp:702] Attempting to truncate the log
>> to 133
>> I0719 12:14:07.323832 11049 coordinator.cpp:348] Coordinator attempting
>> to write TRUNCATE action at position 134
>> I0719 12:14:07.324877 11049 replica.cpp:537] Replica received write
>> request for position 134 from (12)@10.5.7.4:5050
>> I0719 12:14:07.325728 11049 leveldb.cpp:341] Persisting action (18 bytes)
>> to leveldb took 756749ns
>> I0719 12:14:07.325804 11049 replica.cpp:712] Persisted action at 134
>> W0719 12:14:07.326323 11047 master.cpp:6170] Possibly orphaned completed
>> task webapp.42f99194-4dda-11e6-8041-02421568c935 of framework
>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001 that ran on slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com)
>> I0719 12:14:07.326839 11049 replica.cpp:691] Replica received learned
>> notice for position 134 from @0.0.0.0:0
>> I0719 12:14:07.327742 11049 leveldb.cpp:341] Persisting action (20 bytes)
>> to leveldb took 717423ns
>> I0719 12:14:07.327713 11047 master.cpp:4521] Re-registered slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com) with cpus(*):4; mem(*):2929; disk(*):242457;
>> ports(*):[31000-32000]
>> I0719 12:14:07.328063 11047 master.cpp:4556] Sending updated checkpointed
>> resources  to slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@
>> 10.5.7.5:5051 (dev2.something.com)
>> I0719 12:14:07.328157 11049 leveldb.cpp:399] Deleting ~2 keys from
>> leveldb took 188898ns
>> I0719 12:14:07.328233 11049 replica.cpp:712] Persisted action at 134
>> I0719 12:14:07.328300 11049 replica.cpp:697] Replica learned TRUNCATE
>> action at position 134
>> I0719 12:14:07.329198 11046 hierarchical.cpp:473] Added slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) with
>> cpus(*):4; mem(*):2929; disk(*):242457; ports(*):[31000-32000] (allocated: )
>> I0719 12:14:07.330371 11044 master.cpp:4618] Received update of slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com) with total oversubscribed resources
>> I0719 12:14:07.332751 11046 hierarchical.cpp:531] Slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) updated
>> with oversubscribed resources  (total: cpus(*):4; mem(*):2929;
>> disk(*):242457; ports(*):[31000-32000], allocated: cpus(*):4; mem(*):2929;
>> disk(*):242457; ports(*):[31000-32000])
>> I0719 12:14:07.332794 11047 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:07.432186 11045 master.cpp:3641] Processing DECLINE call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O0 ] for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:09.503839 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:14:09.877415 11044 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:09.877856 11044 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:14:09.880434 11051 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:09.886831 11048 master.cpp:3641] Processing DECLINE call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O1 ] for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:14.894800 11045 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:14.895149 11051 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:14:14.897150 11045 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:14.903162 11044 master.cpp:3641] Processing DECLINE call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O2 ] for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:19.714318 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:14:19.914870 11049 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:19.915210 11047 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:14:19.917094 11049 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:19.923250 11051 master.cpp:3641] Processing DECLINE call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O3 ] for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:20.019886 11050 master.cpp:4997] Performing implicit task
>> state reconciliation for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:29.515525 11048 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:29.515820 11048 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:14:29.517138 11048 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:29.766815 11044 master.cpp:3104] Processing ACCEPT call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O4 ] on slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com) for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:29.771697 11049 master.hpp:177] Adding task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>> mem(*):32; ports(*):[31990-31990] on slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
>> I0719 12:14:29.772017 11049 master.cpp:3589] Launching task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 with
>> resources cpus(*):0.1; mem(*):32; ports(*):[31990-31990] on slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com)
>> I0719 12:14:30.589148 11045 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:30.595021 11046 master.cpp:3641] Processing DECLINE call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O5 ] for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:14:30.610872 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:14:30.803971 11048 master.cpp:4763] Status update TASK_RUNNING
>> (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 from slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com)
>> I0719 12:14:30.804188 11048 master.cpp:4811] Forwarding status update
>> TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:14:30.804988 11048 master.cpp:6421] Updating the state of task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_RUNNING,
>> status update state: TASK_RUNNING)
>> I0719 12:14:30.868975 11046 master.cpp:3918] Processing ACKNOWLEDGE call
>> 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f for task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 on slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>> I0719 12:14:40.864478 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:14:51.114728 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:01.365391 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:08.869536 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51290 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:18.887311 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:28.904626 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:38.918462 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:48.932732 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:15:58.946310 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:16:08.961182 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:16:18.976491 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:16:29.148555 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:16:30.951886 11046 master.cpp:5324] Sending 1 offers to
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:16:30.958127 11047 master.cpp:3641] Processing DECLINE call for
>> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O6 ] for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:16:39.398538 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:16:49.648846 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:16:59.899797 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:17:07.350857 11047 master.cpp:242] Shutting down slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 due to health check timeout
>> W0719 12:17:07.351191 11047 master.cpp:4930] Shutting down slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com) with message 'health check timed out'
>> I0719 12:17:07.351920 11047 master.cpp:6201] Removing slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com): health check timed out
>> I0719 12:17:07.352468 11047 master.cpp:6421] Updating the state of task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_LOST, status
>> update state: TASK_LOST)
>> I0719 12:17:07.352746 11045 hierarchical.cpp:505] Removed slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
>> I0719 12:17:07.353837 11047 master.cpp:6487] Removing task
>> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
>> mem(*):32; ports(*):[31990-31990] of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 on slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
>> dev2.something.com)
>> I0719 12:17:07.356307 11046 registrar.cpp:439] Applied 1 operations in
>> 164323ns; attempting to update the 'registry'
>> I0719 12:17:07.360155 11045 log.cpp:683] Attempting to append 184 bytes
>> to the log
>> I0719 12:17:07.360399 11050 coordinator.cpp:348] Coordinator attempting
>> to write APPEND action at position 135
>> I0719 12:17:07.361938 11051 replica.cpp:537] Replica received write
>> request for position 135 from (40)@10.5.7.4:5050
>> I0719 12:17:07.362778 11051 leveldb.cpp:341] Persisting action (204
>> bytes) to leveldb took 728456ns
>> I0719 12:17:07.362862 11051 replica.cpp:712] Persisted action at 135
>> I0719 12:17:07.363806 11051 replica.cpp:691] Replica received learned
>> notice for position 135 from @0.0.0.0:0
>> I0719 12:17:07.364334 11051 leveldb.cpp:341] Persisting action (206
>> bytes) to leveldb took 430978ns
>> I0719 12:17:07.364414 11051 replica.cpp:712] Persisted action at 135
>> I0719 12:17:07.364471 11051 replica.cpp:697] Replica learned APPEND
>> action at position 135
>> I0719 12:17:07.366155 11047 registrar.cpp:484] Successfully updated the
>> 'registry' in 9.689088ms
>> I0719 12:17:07.366540 11051 log.cpp:702] Attempting to truncate the log
>> to 135
>> I0719 12:17:07.366657 11046 master.cpp:6320] Removed slave
>> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com): health
>> check timed out
>> I0719 12:17:07.366739 11047 coordinator.cpp:348] Coordinator attempting
>> to write TRUNCATE action at position 136
>> I0719 12:17:07.366789 11046 master.cpp:4806] Sending status update
>> TASK_LOST for task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 'Slave dev2.something.com
>> removed: health check timed out'
>> I0719 12:17:07.367236 11046 master.cpp:6343] Notifying framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 of lost
>> slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) after
>> recovering
>> I0719 12:17:07.367993 11049 replica.cpp:537] Replica received write
>> request for position 136 from (41)@10.5.7.4:5050
>> I0719 12:17:07.368701 11049 leveldb.cpp:341] Persisting action (18 bytes)
>> to leveldb took 607673ns
>> I0719 12:17:07.368780 11049 replica.cpp:712] Persisted action at 136
>> I0719 12:17:07.369793 11048 replica.cpp:691] Replica received learned
>> notice for position 136 from @0.0.0.0:0
>> I0719 12:17:07.374290 11048 leveldb.cpp:341] Persisting action (20 bytes)
>> to leveldb took 4.333673ms
>> I0719 12:17:07.374531 11048 leveldb.cpp:399] Deleting ~2 keys from
>> leveldb took 142498ns
>> I0719 12:17:07.374651 11048 replica.cpp:712] Persisted action at 136
>> I0719 12:17:07.374763 11048 replica.cpp:697] Replica learned TRUNCATE
>> action at position 136
>> I0719 12:17:07.413800 11048 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:17:07.414114 11051 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:10.107131 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:17:12.435168 11051 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:17:12.435470 11051 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:17.455554 11049 master.cpp:3720] Processing REVIVE call for
>> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:17:17.456195 11044 hierarchical.cpp:988] Removed offer filters
>> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:20.620965 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:17:28.907829 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:28.908550 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:17:30.634392 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:17:33.887810 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:17:33.888640 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:17:33.888864 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:33.889627 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:17:38.890091 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:17:38.890705 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:17:38.890908 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:38.891654 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:17:41.619175 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:17:43.892462 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:17:43.893380 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:17:43.893582 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:43.894261 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:17:48.894907 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:17:48.895597 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:17:48.895742 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:48.896458 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:17:52.622318 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:17:53.897758 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:17:53.898336 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:17:53.898510 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:53.899430 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:17:58.900116 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:17:58.900677 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:17:58.901741 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:17:58.902381 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:03.621628 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:18:03.902536 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:03.903519 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:03.903714 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:03.904594 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:08.904714 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:18:08.905208 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:08.905773 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:08.905913 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:08.906513 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:13.907012 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:18:13.907600 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:13.907958 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:13.908123 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:13.908871 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:14.624475 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:18:18.909356 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:18.910174 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:18.910326 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:18.911255 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:23.911778 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:18:23.912201 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:23.912739 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:23.913128 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:23.913684 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:25.624198 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:18:28.914320 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:28.914964 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:28.915148 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:28.915825 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:33.916498 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:33.917145 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:33.917282 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:33.918520 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:36.625072 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:18:38.918712 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:38.919631 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:38.920052 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:38.920876 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:43.921413 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:43.922125 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:43.922269 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:43.923209 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:47.627984 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:18:48.923180 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:48.923815 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:48.924010 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:48.924589 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:53.925963 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:53.926895 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:53.927124 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:53.927942 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:18:58.626113 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:18:58.927940 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:18:58.928613 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:18:58.928766 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:18:58.930127 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:03.930527 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:03.931396 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:03.931637 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:03.932543 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:08.932302 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:19:08.932852 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:08.933034 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:08.933212 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:08.933835 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:09.627965 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:19:13.934926 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:13.935514 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:13.935757 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:13.936638 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:18.936506 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:19:18.937036 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:18.937420 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:18.937613 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:18.938341 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:20.624949 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:19:23.938432 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:23.939040 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:23.939214 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:23.940078 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:28.941543 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:28.942198 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:28.942435 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:28.943140 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:31.627547 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:19:33.943943 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:19:33.944599 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:33.944835 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:33.944988 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:33.945693 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:38.945986 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:19:38.946478 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:38.946812 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:38.946949 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:38.947438 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:42.628363 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:19:43.948642 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:43.949349 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:43.949550 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:43.950319 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:48.951385 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:48.951884 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:48.952000 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:48.952672 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:53.628947 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:19:53.954244 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:53.954866 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:53.955090 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:53.956058 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:19:58.956197 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:19:58.956797 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:19:58.957006 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:19:58.957918 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:03.958680 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:03.959282 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:03.959450 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:03.960024 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:04.630003 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:20:08.961760 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:20:08.962337 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:08.962745 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:08.962887 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:08.963418 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:13.964138 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:13.964747 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:13.965031 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:13.965772 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:15.630905 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:20:18.966778 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:18.967433 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:18.967687 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:18.968503 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:23.969029 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:23.969832 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:23.969976 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:23.970935 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:26.628504 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:20:28.971153 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:20:28.971578 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:28.971874 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:28.972044 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:28.972594 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:33.973855 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:20:33.974304 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:33.974576 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:33.974689 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:33.975183 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:37.629881 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:20:38.976158 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:38.977018 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:38.977200 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:38.978510 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:43.977927 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:43.978523 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:43.978688 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:43.979673 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:48.631947 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:20:48.980355 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:48.981098 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:48.981258 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:48.982522 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:53.983012 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:53.983757 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:53.983986 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:53.984787 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:58.985410 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:20:58.985873 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:20:58.986018 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:20:58.986199 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:20:58.986820 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:20:59.630745 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:21:03.987254 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:03.987884 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:03.988030 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:03.989013 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:08.989428 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:21:08.990020 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:08.990285 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:08.990468 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:08.990981 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:10.633591 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:21:13.992058 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:13.992918 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:13.993191 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:13.994236 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:18.994022 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:21:18.994477 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:18.994962 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:18.995141 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:18.995689 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:21.647516 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:21:23.996325 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:23.997014 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:23.997248 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:23.998145 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:28.998471 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:28.999652 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:28.999872 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:29.000628 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:32.635963 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:21:34.000746 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:34.001461 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:34.001718 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:34.002413 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:39.003231 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:39.003926 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:39.004137 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:39.004909 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:43.639329 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:21:44.005383 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:44.005898 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:44.006666 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:44.007246 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:49.007254 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:49.007946 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:49.008180 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:49.009217 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:54.009101 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:54.009840 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:54.010089 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:54.010777 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:21:54.634322 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:21:59.011060 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:21:59.011662 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:21:59.011852 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:21:59.012444 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:04.013264 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:22:04.013864 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:04.014137 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:04.014339 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:04.015051 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:05.633569 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:22:09.015625 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:09.016332 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:09.016549 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:09.017400 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:14.018102 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:14.018810 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:14.018975 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:14.020356 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:16.638113 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:22:19.020917 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:22:19.021430 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:19.021756 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:19.021895 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:19.022616 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:24.023571 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:24.024431 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:24.024754 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:24.026060 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:27.643009 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:22:29.026162 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:29.026885 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:29.027034 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:29.028298 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:34.028923 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:34.029616 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:34.029865 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:34.030767 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:38.638548 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:22:39.031050 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:39.031689 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:39.031838 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:39.032711 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:44.033393 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:44.034443 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:44.034685 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:44.035321 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:49.035828 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:49.036345 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:49.036553 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:49.037240 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:49.636526 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:22:54.038525 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:22:54.039108 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:54.039549 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:54.039691 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:54.040293 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:22:59.040702 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:22:59.041216 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:22:59.041524 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:22:59.041749 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:22:59.042407 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:00.639822 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:23:04.042881 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:04.043432 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:04.043550 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:04.044637 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:23:09.044903 11051 master.cpp:5541] Authentication timed out
>> W0719 12:23:09.045856 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:09.046164 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:09.046769 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:11.640506 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:23:14.047653 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:14.048249 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:14.048429 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:14.049170 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:19.049494 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:19.050199 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:19.050398 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:19.051105 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:22.639598 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:23:24.051370 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:24.052043 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:24.052255 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:24.053045 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:29.053254 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:23:29.053788 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:29.054085 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:29.054247 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:29.054867 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:33.639268 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:23:34.055438 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:23:34.055974 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:34.056213 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:34.056349 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:34.056926 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:39.057729 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:39.058291 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:39.058435 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:39.059099 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:44.059839 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:44.060503 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:44.060667 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:44.061758 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:44.639904 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:23:49.062311 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:49.062969 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:49.063107 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:49.063963 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:54.064702 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:54.065510 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:54.065732 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:54.066545 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:23:55.642850 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:23:59.067154 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:23:59.067770 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:23:59.068084 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:23:59.068683 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:04.069491 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:04.070219 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:04.070417 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:04.071239 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:06.643237 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:24:09.071045 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:09.071583 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:09.071717 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:09.072408 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:14.073580 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:14.074219 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:14.074439 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:14.075234 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:17.645912 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:24:19.076020 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:19.076668 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:19.077389 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:19.078135 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:20.012001 11051 master.cpp:5062] Performing explicit task
>> state reconciliation for 1 tasks of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:24:20.012976 11051 master.cpp:4997] Performing implicit task
>> state reconciliation for framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
>> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
>> I0719 12:24:24.078232 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:24.078850 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:24.079015 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:24.079833 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:28.644161 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:24:29.080739 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:29.081439 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:29.081719 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:29.082607 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:34.083586 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:24:34.084177 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:34.084424 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:34.085206 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:24:39.085710 11044 master.cpp:5541] Authentication timed out
>> W0719 12:24:39.085863 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:39.086200 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:39.086688 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:39.645274 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:24:44.087625 11051 master.cpp:5541] Authentication timed out
>> W0719 12:24:44.087796 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:44.088065 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:44.088768 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:24:49.089455 11046 master.cpp:5541] Authentication timed out
>> W0719 12:24:49.089862 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:49.090831 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:49.091661 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:24:50.645830 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:24:54.092033 11045 master.cpp:5541] Authentication timed out
>> W0719 12:24:54.092311 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:54.093353 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:54.093953 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:24:59.094514 11045 master.cpp:5541] Authentication timed out
>> W0719 12:24:59.094877 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:24:59.095696 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:24:59.096318 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:01.646898 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:25:04.098826 11045 master.cpp:5541] Authentication timed out
>> I0719 12:25:04.099218 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:25:04.099647 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:04.099804 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:04.100505 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:25:09.101160 11048 master.cpp:5541] Authentication timed out
>> W0719 12:25:09.101491 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:09.101846 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:09.102602 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:11.660732 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:25:14.102864 11046 master.cpp:5541] Authentication timed out
>> W0719 12:25:14.103103 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:14.103487 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:14.104300 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:25:19.104631 11046 master.cpp:5541] Authentication timed out
>> W0719 12:25:19.104817 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:19.106343 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:19.106928 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:21.673857 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:25:24.107980 11050 master.cpp:5541] Authentication timed out
>> W0719 12:25:24.108252 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:24.109048 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:24.109756 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:25:29.110285 11046 master.cpp:5541] Authentication timed out
>> W0719 12:25:29.110486 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:29.111874 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:29.112534 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:32.648926 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:25:34.113827 11046 master.cpp:5541] Authentication timed out
>> W0719 12:25:34.114099 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:34.114449 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:34.115272 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> W0719 12:25:39.115715 11048 master.cpp:5541] Authentication timed out
>> I0719 12:25:39.116320 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:25:39.116565 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:39.116698 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:39.117486 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:43.649667 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> W0719 12:25:44.117769 11050 master.cpp:5541] Authentication timed out
>> W0719 12:25:44.118093 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:44.119288 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:44.119913 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:49.121246 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:25:49.122274 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:49.122517 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:49.123277 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:54.123177 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:25:54.123744 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:54.123908 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:54.124613 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:25:54.650367 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:25:59.125444 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:25:59.126139 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:25:59.126317 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:25:59.127210 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:04.127732 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:04.128376 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:04.128587 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:04.129299 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:05.650473 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:26:09.129295 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:09.129942 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:09.130143 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:09.130893 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:14.132257 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:14.132984 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:14.133221 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:14.133903 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:16.650934 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:26:19.134830 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:26:19.135447 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:19.135695 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:19.135893 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:19.136826 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:24.137058 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:26:24.137684 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:24.137899 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:24.138236 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:24.138895 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:27.653244 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:26:29.139055 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:26:29.139578 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:29.139806 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:29.139937 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:29.140700 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:34.141217 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:34.142405 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:34.142554 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:34.143790 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:38.652467 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:26:39.143749 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:39.144413 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:39.144654 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:39.145241 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:44.146155 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:44.146859 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:44.147155 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:44.147868 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:49.148578 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:49.149238 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:49.149387 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:49.150285 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:49.652683 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:26:54.150988 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:26:54.151628 11047 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:54.152030 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:54.152228 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:54.152897 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:26:59.153410 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:26:59.153934 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:26:59.154147 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:26:59.154831 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:00.653033 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:27:04.155833 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:04.156525 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:04.156755 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:04.157620 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:09.158335 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:09.158910 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:09.159360 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:09.159893 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:11.654698 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:27:14.160549 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:14.161084 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:14.161367 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:14.162240 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:19.162997 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:19.163769 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:19.163976 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:19.165292 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:22.657127 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:27:24.165325 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:27:24.165812 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:24.166124 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:24.166335 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:24.166985 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:29.167835 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:29.168339 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:29.168457 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:29.169409 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:33.654701 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:27:34.170714 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:34.171576 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:34.171741 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:34.172905 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:39.173723 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:39.174795 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:39.174945 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:39.175669 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:44.175698 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:44.176259 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:44.176416 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:44.177232 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:44.655261 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:27:49.178457 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:27:49.178927 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:49.179082 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:49.179266 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:49.179967 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:54.181052 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:54.181738 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:54.181941 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:54.182667 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:27:55.656306 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:27:59.182857 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:27:59.183506 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:27:59.183732 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:27:59.184350 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:04.185437 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:04.186125 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:04.186264 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:04.187189 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:06.656854 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:28:09.188138 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:09.188856 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:09.189314 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:09.190248 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:14.190521 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:14.191016 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:14.191254 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:14.191875 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:17.659039 11047 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:28:19.192914 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:28:19.193475 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:19.193802 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:19.193939 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:19.194689 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:24.195497 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:24.196179 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:24.196390 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:24.197120 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:28.658488 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:28:29.198254 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:28:29.198724 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:29.198997 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:29.199115 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:29.199853 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:34.200729 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:28:34.201292 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:34.201460 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:34.201683 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:34.202527 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:39.202571 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:39.203177 11045 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:39.203344 11045 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:39.204205 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:39.659113 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:28:44.204735 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:44.205322 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:44.205466 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:44.206336 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:49.206876 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:28:49.207523 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:49.208042 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:49.208204 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:49.208876 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:50.659682 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:28:54.208958 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:54.209754 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:54.209985 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:54.210769 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:28:59.211021 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:28:59.211598 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:28:59.211750 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:28:59.212461 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:01.659540 11046 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:29:04.212937 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:04.213676 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:04.213832 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:04.214392 11051 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:09.215848 11050 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:09.216425 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:09.216625 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:09.217329 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:12.663728 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:29:14.218930 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:14.219707 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:14.220160 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:14.220715 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:19.221014 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:19.221758 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:19.221928 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:19.222981 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:23.660816 11048 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:29:24.223538 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:24.224192 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:24.224408 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:24.225181 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:29.225445 11045 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:29.226102 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:29.226274 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:29.227026 11050 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:34.227371 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:34.227906 11050 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:34.228116 11050 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:34.228906 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:34.662215 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:29:39.230406 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:39.231156 11051 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:39.231384 11051 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:39.232285 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:44.232631 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:44.233209 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:44.233371 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:44.234177 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:45.662411 11051 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:29:49.234805 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:49.235468 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:49.235636 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:49.237004 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:54.236991 11049 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:54.237576 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:54.237751 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:54.238709 11045 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:29:56.663393 11045 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:29:59.239316 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:29:59.239837 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:29:59.240001 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:29:59.240638 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:04.241629 11044 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:04.242250 11044 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:04.242472 11044 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:04.243203 11044 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:07.677323 11050 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:30:09.244133 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:09.244830 11046 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:09.245038 11046 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:09.245756 11046 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:14.246692 11046 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:14.247503 11049 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:14.247711 11049 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:14.248601 11049 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:18.664516 11044 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:30:19.248931 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:19.249675 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:19.249899 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:19.250809 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:24.250947 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:30:24.251588 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:24.251802 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:24.252038 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:24.252806 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:29.253676 11051 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:29.254382 11047 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:29.254613 11047 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:29.255362 11047 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0719 12:30:29.665109 11049 http.cpp:312] HTTP GET for /master/state from
>> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
>> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
>> I0719 12:30:34.255038 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> I0719 12:30:34.255576 11048 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0719 12:30:34.255727 11048 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>> I0719 12:30:34.255898 11048 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0719 12:30:34.256418 11048 authenticator.cpp:98] Creating new server
>> SASL connection
>>
>>
>>
>>
>> AGENT:
>>
>> Log file created at: 2016/07/19 12:17:26
>> Running on machine: dev2
>> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
>> I0719 12:17:26.442286  2706 logging.cpp:188] INFO level logging started!
>> I0719 12:17:26.442948  2706 main.cpp:223] Build: 2016-07-08 16:57:43 by
>> root
>> I0719 12:17:26.443025  2706 main.cpp:225] Version: 0.28.2
>> I0719 12:17:26.563817  2706 containerizer.cpp:149] Using isolation:
>> posix/cpu,posix/mem,filesystem/posix
>> I0719 12:17:26.577617  2706 linux_launcher.cpp:101] Using
>> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
>> I0719 12:17:26.587507  2706 main.cpp:328] Starting Mesos slave
>> I0719 12:17:26.591217  2726 slave.cpp:193] Slave started on 1)@
>> 10.5.7.5:5051
>> I0719 12:17:26.591284  2726 slave.cpp:194] Flags at startup:
>> --appc_simple_discovery_uri_prefix="http://"
>> --appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5"
>> --cgroups_cpu_enable_pids_and_tids_count="false"
>> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
>> --cgroups_limit_swap="false" --cgroups_root="mesos"
>> --container_disk_watch_interval="15secs" --containerizers="docker,mesos"
>> --credential="/etc/mesos/auth/credential" --default_role="*"
>> --disk_watch_interval="1mins" --docker="docker"
>> --docker_kill_orphans="true" --docker_registry="
>> https://registry-1.docker.io" --docker_remove_delay="6hrs"
>> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
>> --docker_store_dir="/tmp/mesos/store/docker"
>> --enforce_container_disk_quota="false"
>> --executor_environment_variables="{"SSL_CERT_FILE":"\/etc\/mesos\/ssl\/mesos.pem","SSL_ENABLED":"true","SSL_KEY_FILE":"\/etc\/mesos\/ssl\/mesos.key"}"
>> --executor_registration_timeout="5mins"
>> --executor_shutdown_grace_period="5secs"
>> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
>> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
>> --hadoop_home="" --help="false" --hostname_lookup="true"
>> --image_provisioner_backend="copy" --initialize_driver_logging="true"
>> --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos"
>> --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO"
>> --master="zk://10.5.7.4:2181/mesos"
>> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
>> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
>> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
>> --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true"
>> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
>> --switch_user="true" --systemd_enable_support="true"
>> --systemd_runtime_directory="/run/systemd/system" --version="false"
>> --work_dir="/var/lib/mesos"
>> I0719 12:17:26.592231  2726 credentials.hpp:83] Loading credential for
>> authentication from '/etc/mesos/auth/credential'
>> W0719 12:17:26.592401  2726 credentials.hpp:98] Permissions on credential
>> file '/etc/mesos/auth/credential' are too open. It is recommended that your
>> credential file is NOT accessible by others.
>> I0719 12:17:26.593298  2730 group.cpp:349] Group process (group(1)@
>> 10.5.7.5:5051) connected to ZooKeeper
>> I0719 12:17:26.593535  2730 group.cpp:831] Syncing group operations:
>> queue size (joins, cancels, datas) = (0, 0, 0)
>> I0719 12:17:26.593608  2730 group.cpp:427] Trying to create path '/mesos'
>> in ZooKeeper
>> I0719 12:17:26.600158  2725 detector.cpp:152] Detected a new leader:
>> (id='2')
>> I0719 12:17:26.600987  2727 group.cpp:700] Trying to get
>> '/mesos/json.info_0000000002' in ZooKeeper
>> I0719 12:17:26.603498  2726 slave.cpp:324] Slave using credential for: dev
>> I0719 12:17:26.604095  2731 detector.cpp:479] A new leading master (UPID=
>> master@10.5.7.4:5050) is detected
>> I0719 12:17:26.605425  2726 slave.cpp:464] Slave resources: cpus(*):4;
>> mem(*):2929; disk(*):242457; ports(*):[31000-32000]
>> I0719 12:17:26.605607  2726 slave.cpp:472] Slave attributes: [  ]
>> I0719 12:17:26.605659  2726 slave.cpp:477] Slave hostname:
>> dev2.something.com
>> I0719 12:17:26.612834  2729 state.cpp:58] Recovering state from
>> '/var/lib/mesos/meta'
>> I0719 12:17:26.630897  2731 slave.cpp:4638] Recovering framework
>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
>> I0719 12:17:26.631350  2731 slave.cpp:5461] Recovering executor
>> 'webapp.42f99194-4dda-11e6-8041-02421568c935' of framework
>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
>> I0719 12:17:26.633368  2728 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935/runs/7b4fc655-5680-4581-8025-5c218de5e893'
>> for gc 6.99456443088days in the future
>> I0719 12:17:26.633872  2731 slave.cpp:4063] Cleaning up framework
>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
>> I0719 12:17:26.633875  2728 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935/runs/7b4fc655-5680-4581-8025-5c218de5e893'
>> for gc 6.99456442714074days in the future
>> I0719 12:17:26.634140  2728 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935'
>> for gc 6.99456442632days in the future
>> I0719 12:17:26.634322  2731 slave.cpp:4638] Recovering framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:26.634459  2728 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935'
>> for gc 6.99456442578667days in the future
>> I0719 12:17:26.634485  2731 slave.cpp:5461] Recovering executor
>> 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:26.634722  2728 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001'
>> for gc 6.9999926607763days in the future
>> I0719 12:17:26.634914  2728 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001'
>> for gc 6.99999265988444days in the future
>> I0719 12:17:26.634171  2729 status_update_manager.cpp:282] Closing status
>> update streams for framework b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
>> I0719 12:17:26.637401  2727 status_update_manager.cpp:200] Recovering
>> status update manager
>> I0719 12:17:26.637521  2727 status_update_manager.cpp:208] Recovering
>> executor 'webapp.42f99194-4dda-11e6-8041-02421568c935' of framework
>> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
>> I0719 12:17:26.637789  2727 status_update_manager.cpp:208] Recovering
>> executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:26.642676  2726 docker.cpp:773] Recovering Docker containers
>> I0719 12:17:26.643991  2731 containerizer.cpp:407] Recovering
>> containerizer
>> I0719 12:17:26.647322  2731 containerizer.cpp:456] Skipping recovery of
>> executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 because it was not launched from
>> mesos containerizer
>> I0719 12:17:26.654330  2727 provisioner.cpp:245] Provisioner recovery
>> complete
>> I0719 12:17:26.877194  2726 docker.cpp:877] Recovering container
>> '65fa6566-806b-407f-a273-ef3b94f6dede' for executor
>> 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
>> 'ee03b9d8-797a-421b-9542-652e53fbc4fb-0000'
>> I0719 12:17:26.883574  2727 slave.cpp:4490] Sending reconnect request to
>> executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 at executor(1)@10.5.7.5:51307
>> I0719 12:17:26.900796  2728 slave.cpp:2799] Re-registering executor
>> 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
>> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
>> I0719 12:17:26.993151  2729 docker.cpp:1427] Updated 'cpu.shares' to 204
>> at
>> /sys/fs/cgroup/cpu/docker/9efb4b79f889d69111e33dbce67bd62f8f82f3068d76a0b53118040f00f1ea5e
>> for container 65fa6566-806b-407f-a273-ef3b94f6dede
>> I0719 12:17:26.994645  2729 docker.cpp:1462] Updated
>> 'memory.soft_limit_in_bytes' to 64MB for container
>> 65fa6566-806b-407f-a273-ef3b94f6dede
>> I0719 12:17:26.997007  2729 docker.cpp:1488] Updated
>> 'memory.limit_in_bytes' to 64MB at
>> /sys/fs/cgroup/memory/docker/9efb4b79f889d69111e33dbce67bd62f8f82f3068d76a0b53118040f00f1ea5e
>> for container 65fa6566-806b-407f-a273-ef3b94f6dede
>> I0719 12:17:28.886029  2725 slave.cpp:2952] Cleaning up un-reregistered
>> executors
>> I0719 12:17:28.886433  2725 slave.cpp:4550] Finished recovery
>> I0719 12:17:28.887312  2725 slave.cpp:4583] Garbage collecting old slave
>> 0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0
>> I0719 12:17:28.887639  2725 slave.cpp:4583] Garbage collecting old slave
>> 6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0
>> I0719 12:17:28.887648  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0' for gc
>> 6.99998972913185days in the future
>> I0719 12:17:28.887792  2725 slave.cpp:4583] Garbage collecting old slave
>> 31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1
>> I0719 12:17:28.887933  2725 slave.cpp:4583] Garbage collecting old slave
>> 9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0
>> I0719 12:17:28.887967  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0' for gc
>> 6.99998972714074days in the future
>> I0719 12:17:28.888082  2725 slave.cpp:4583] Garbage collecting old slave
>> ca647578-5569-4990-bce2-2bf86ffacde3-S0
>> I0719 12:17:28.888191  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0' for gc
>> 6.99998972571852days in the future
>> I0719 12:17:28.888355  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0' for gc
>> 6.99998972497778days in the future
>> I0719 12:17:28.888499  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1' for gc
>> 6.99998972407407days in the future
>> I0719 12:17:28.888689  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1' for gc
>> 6.99998972341333days in the future
>> I0719 12:17:28.888829  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0' for gc
>> 6.99998972240593days in the future
>> I0719 12:17:28.888967  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0' for gc
>> 6.99998972164444days in the future
>> I0719 12:17:28.889101  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/slaves/ca647578-5569-4990-bce2-2bf86ffacde3-S0' for gc
>> 6.99998972040593days in the future
>> I0719 12:17:28.889317  2732 gc.cpp:55] Scheduling
>> '/var/lib/mesos/meta/slaves/ca647578-5569-4990-bce2-2bf86ffacde3-S0' for gc
>> 6.99998971916741days in the future
>> I0719 12:17:28.889744  2732 status_update_manager.cpp:174] Pausing
>> sending status updates
>> I0719 12:17:28.889770  2725 slave.cpp:796] New master detected at
>> master@10.5.7.4:5050
>> I0719 12:17:28.909875  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:28.909971  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:28.910511  2725 slave.cpp:832] Detecting new master
>> I0719 12:17:28.911711  2732 authenticatee.cpp:97] Initializing client SASL
>> I0719 12:17:28.914371  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:17:33.911911  2731 slave.cpp:944] Authentication timed out
>> W0719 12:17:33.912632  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:17:33.912788  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:33.912845  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:33.913310  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:17:38.914300  2730 slave.cpp:944] Authentication timed out
>> W0719 12:17:38.915067  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:17:38.915307  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:38.915366  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:38.915830  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:17:43.916915  2725 slave.cpp:944] Authentication timed out
>> W0719 12:17:43.917677  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:17:43.917862  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:43.917974  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:43.918507  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:17:48.918875  2730 slave.cpp:944] Authentication timed out
>> W0719 12:17:48.919649  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:17:48.919847  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:48.919919  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:48.920433  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:17:53.921869  2727 slave.cpp:944] Authentication timed out
>> W0719 12:17:53.922817  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:17:53.922976  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:53.923032  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:53.923563  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:17:58.924680  2732 slave.cpp:944] Authentication timed out
>> W0719 12:17:58.925246  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:17:58.925434  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:17:58.925498  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:17:58.926103  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:03.927134  2730 slave.cpp:944] Authentication timed out
>> W0719 12:18:03.927891  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:03.928052  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:03.928112  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:03.928536  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:08.929105  2728 slave.cpp:944] Authentication timed out
>> W0719 12:18:08.929832  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:08.930012  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:08.930093  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:08.930577  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:13.931543  2729 slave.cpp:944] Authentication timed out
>> W0719 12:18:13.932309  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:13.932477  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:13.932550  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:13.933042  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:18.933753  2731 slave.cpp:944] Authentication timed out
>> W0719 12:18:18.934530  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:18.934729  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:18.934792  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:18.935309  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:23.936511  2729 slave.cpp:944] Authentication timed out
>> W0719 12:18:23.937216  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:23.937366  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:23.937425  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:23.937896  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:18:26.607225  2729 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169253812823067days
>> W0719 12:18:28.938818  2726 slave.cpp:944] Authentication timed out
>> W0719 12:18:28.939522  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:28.939715  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:28.939811  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:28.940253  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:33.941254  2725 slave.cpp:944] Authentication timed out
>> W0719 12:18:33.941874  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:33.942029  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:33.942085  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:33.942587  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:38.943189  2728 slave.cpp:944] Authentication timed out
>> W0719 12:18:38.943855  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:38.944048  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:38.944110  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:38.944572  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:43.945793  2725 slave.cpp:944] Authentication timed out
>> W0719 12:18:43.946396  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:43.946601  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:43.946729  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:43.947180  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:48.948164  2731 slave.cpp:944] Authentication timed out
>> W0719 12:18:48.948698  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:48.948873  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:48.948927  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:48.949345  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:53.950121  2727 slave.cpp:944] Authentication timed out
>> W0719 12:18:53.950748  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:53.951323  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:53.951418  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:53.951922  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:18:58.952540  2727 slave.cpp:944] Authentication timed out
>> W0719 12:18:58.953218  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:18:58.953387  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:18:58.953461  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:18:58.953919  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:03.954779  2729 slave.cpp:944] Authentication timed out
>> W0719 12:19:03.955401  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:03.955580  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:03.955657  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:03.956127  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:08.957002  2727 slave.cpp:944] Authentication timed out
>> W0719 12:19:08.957651  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:08.957834  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:08.957892  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:08.958351  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:13.959039  2731 slave.cpp:944] Authentication timed out
>> W0719 12:19:13.959734  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:13.959925  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:13.959991  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:13.960530  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:18.961040  2730 slave.cpp:944] Authentication timed out
>> W0719 12:19:18.961683  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:18.961946  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:18.962019  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:18.962450  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:23.963018  2725 slave.cpp:944] Authentication timed out
>> W0719 12:19:23.963678  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:23.963838  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:23.963894  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:23.964344  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:19:26.608978  2729 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169253371042454days
>> W0719 12:19:28.965499  2726 slave.cpp:944] Authentication timed out
>> W0719 12:19:28.966660  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:28.966851  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:28.966958  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:28.967434  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:33.968541  2727 slave.cpp:944] Authentication timed out
>> W0719 12:19:33.969198  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:33.969369  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:33.969426  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:33.969883  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:38.970623  2732 slave.cpp:944] Authentication timed out
>> W0719 12:19:38.971215  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:38.971395  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:38.971454  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:38.971917  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:43.973112  2731 slave.cpp:944] Authentication timed out
>> W0719 12:19:43.973659  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:43.973954  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:43.974014  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:43.974584  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:48.975668  2731 slave.cpp:944] Authentication timed out
>> W0719 12:19:48.976529  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:48.976691  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:48.976758  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:48.977280  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:53.978732  2725 slave.cpp:944] Authentication timed out
>> W0719 12:19:53.979405  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:53.979600  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:53.979676  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:53.980168  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:19:58.980532  2726 slave.cpp:944] Authentication timed out
>> W0719 12:19:58.981240  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:19:58.981444  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:19:58.981531  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:19:58.982074  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:03.983041  2732 slave.cpp:944] Authentication timed out
>> W0719 12:20:03.983832  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:03.984200  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:03.984313  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:03.984802  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:08.986167  2726 slave.cpp:944] Authentication timed out
>> W0719 12:20:08.986836  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:08.986984  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:08.987040  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:08.987558  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:13.988708  2731 slave.cpp:944] Authentication timed out
>> W0719 12:20:13.989320  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:13.989475  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:13.989531  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:13.989976  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:18.991140  2729 slave.cpp:944] Authentication timed out
>> W0719 12:20:18.991741  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:18.991935  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:18.992003  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:18.992486  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:23.993616  2732 slave.cpp:944] Authentication timed out
>> W0719 12:20:23.994303  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:23.994451  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:23.994508  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:23.994971  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:20:26.610416  2725 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169252929261840days
>> W0719 12:20:28.995944  2725 slave.cpp:944] Authentication timed out
>> W0719 12:20:28.996537  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:28.996763  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:28.996839  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:28.997261  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:33.998452  2726 slave.cpp:944] Authentication timed out
>> W0719 12:20:33.999094  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:33.999261  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:33.999356  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:33.999822  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:39.000432  2726 slave.cpp:944] Authentication timed out
>> W0719 12:20:39.001195  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:39.001364  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:39.001435  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:39.001938  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:44.002457  2732 slave.cpp:944] Authentication timed out
>> W0719 12:20:44.003163  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:44.003326  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:44.003382  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:44.003878  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:49.004884  2731 slave.cpp:944] Authentication timed out
>> W0719 12:20:49.005506  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:49.005669  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:49.005765  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:49.006237  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:54.007592  2726 slave.cpp:944] Authentication timed out
>> W0719 12:20:54.008363  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:54.008564  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:54.008625  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:54.009052  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:20:59.009913  2727 slave.cpp:944] Authentication timed out
>> W0719 12:20:59.010506  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:20:59.010984  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:20:59.011035  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:20:59.011549  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:04.012027  2725 slave.cpp:944] Authentication timed out
>> W0719 12:21:04.012603  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:04.012756  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:04.012812  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:04.013241  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:09.013691  2727 slave.cpp:944] Authentication timed out
>> W0719 12:21:09.014644  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:09.014842  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:09.014914  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:09.015367  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:14.016613  2727 slave.cpp:944] Authentication timed out
>> W0719 12:21:14.017313  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:14.017554  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:14.017611  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:14.018029  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:19.018718  2731 slave.cpp:944] Authentication timed out
>> W0719 12:21:19.019327  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:19.019505  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:19.019610  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:19.020059  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:24.020687  2729 slave.cpp:944] Authentication timed out
>> W0719 12:21:24.021451  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:24.021648  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:24.021708  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:24.022183  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:21:26.611302  2729 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169252045700625days
>> W0719 12:21:29.023134  2732 slave.cpp:944] Authentication timed out
>> W0719 12:21:29.023824  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:29.023972  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:29.024029  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:29.024477  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:34.024953  2730 slave.cpp:944] Authentication timed out
>> W0719 12:21:34.025951  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:34.026146  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:34.026221  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:34.026741  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:39.027717  2731 slave.cpp:944] Authentication timed out
>> W0719 12:21:39.028527  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:39.028733  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:39.028796  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:39.029225  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:44.029718  2732 slave.cpp:944] Authentication timed out
>> W0719 12:21:44.030495  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:44.030712  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:44.030787  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:44.031285  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:49.031599  2726 slave.cpp:944] Authentication timed out
>> W0719 12:21:49.032201  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:49.032347  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:49.032402  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:49.033093  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:54.033625  2732 slave.cpp:944] Authentication timed out
>> W0719 12:21:54.034369  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:54.034518  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:54.034577  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:54.035073  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:21:59.035948  2729 slave.cpp:944] Authentication timed out
>> W0719 12:21:59.036551  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:21:59.036703  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:21:59.036761  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:21:59.037250  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:04.037871  2729 slave.cpp:944] Authentication timed out
>> W0719 12:22:04.038518  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:04.038727  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:04.038805  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:04.039253  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:09.040071  2726 slave.cpp:944] Authentication timed out
>> W0719 12:22:09.040676  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:09.040865  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:09.040998  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:09.041527  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:14.042394  2731 slave.cpp:944] Authentication timed out
>> W0719 12:22:14.043166  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:14.043345  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:14.043403  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:14.043896  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:19.045148  2731 slave.cpp:944] Authentication timed out
>> W0719 12:22:19.045684  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:19.046054  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:19.046166  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:19.046763  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:24.048012  2730 slave.cpp:944] Authentication timed out
>> W0719 12:22:24.048894  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:24.049060  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:24.049118  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:24.049540  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:22:26.612486  2730 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169251935255463days
>> W0719 12:22:29.050376  2732 slave.cpp:944] Authentication timed out
>> W0719 12:22:29.051234  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:29.051421  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:29.051477  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:29.051931  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:34.053143  2730 slave.cpp:944] Authentication timed out
>> W0719 12:22:34.053758  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:34.054283  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:34.054349  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:34.054812  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:39.055835  2725 slave.cpp:944] Authentication timed out
>> W0719 12:22:39.056479  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:39.056671  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:39.056728  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:39.057149  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:44.057876  2727 slave.cpp:944] Authentication timed out
>> W0719 12:22:44.058470  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:44.058732  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:44.058861  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:44.059375  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:49.060206  2728 slave.cpp:944] Authentication timed out
>> W0719 12:22:49.061079  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:49.061347  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:49.061414  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:49.061935  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:54.062958  2729 slave.cpp:944] Authentication timed out
>> W0719 12:22:54.063654  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:54.063827  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:54.063910  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:54.064379  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:22:59.065388  2726 slave.cpp:944] Authentication timed out
>> W0719 12:22:59.065942  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:22:59.066133  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:22:59.066200  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:22:59.066651  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:04.067623  2726 slave.cpp:944] Authentication timed out
>> W0719 12:23:04.068518  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:04.068675  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:04.068732  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:04.069146  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:09.069924  2728 slave.cpp:944] Authentication timed out
>> W0719 12:23:09.070524  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:09.071009  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:09.071141  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:09.071633  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:14.072458  2726 slave.cpp:944] Authentication timed out
>> W0719 12:23:14.073009  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:14.073253  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:14.073353  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:14.073799  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:19.074239  2730 slave.cpp:944] Authentication timed out
>> W0719 12:23:19.074939  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:19.075140  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:19.075202  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:19.075644  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:24.076045  2731 slave.cpp:944] Authentication timed out
>> W0719 12:23:24.076644  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:24.076799  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:24.076855  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:24.077338  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:23:26.613450  2731 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169251714365162days
>> W0719 12:23:29.077744  2732 slave.cpp:944] Authentication timed out
>> W0719 12:23:29.078488  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:29.078699  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:29.078765  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:29.079268  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:34.079994  2726 slave.cpp:944] Authentication timed out
>> W0719 12:23:34.080622  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:34.080775  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:34.080831  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:34.081279  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:39.082494  2732 slave.cpp:944] Authentication timed out
>> W0719 12:23:39.083204  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:39.083395  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:39.083456  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:39.083904  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:44.084430  2728 slave.cpp:944] Authentication timed out
>> W0719 12:23:44.085264  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:44.085417  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:44.085474  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:44.085904  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:49.086913  2732 slave.cpp:944] Authentication timed out
>> W0719 12:23:49.087604  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:49.087771  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:49.087828  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:49.088310  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:54.089237  2730 slave.cpp:944] Authentication timed out
>> W0719 12:23:54.089737  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:54.089936  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:54.090013  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:54.090585  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:23:59.091807  2729 slave.cpp:944] Authentication timed out
>> W0719 12:23:59.092536  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:23:59.092717  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:23:59.092798  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:23:59.093231  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:04.094005  2730 slave.cpp:944] Authentication timed out
>> W0719 12:24:04.094712  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:04.094916  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:04.094985  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:04.095445  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:09.095959  2731 slave.cpp:944] Authentication timed out
>> W0719 12:24:09.096580  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:09.096776  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:09.096835  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:09.097257  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:14.098148  2727 slave.cpp:944] Authentication timed out
>> W0719 12:24:14.098824  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:14.099046  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:14.099112  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:14.099601  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:19.100690  2725 slave.cpp:944] Authentication timed out
>> W0719 12:24:19.101272  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:19.101482  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:19.101557  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:19.102108  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:24.102840  2731 slave.cpp:944] Authentication timed out
>> W0719 12:24:24.103386  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:24.103564  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:24.103623  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:24.104046  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:24:26.614562  2729 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169251272584549days
>> W0719 12:24:29.105397  2726 slave.cpp:944] Authentication timed out
>> W0719 12:24:29.105924  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:29.106081  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:29.106142  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:29.106680  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:34.108180  2727 slave.cpp:944] Authentication timed out
>> W0719 12:24:34.108861  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:34.109020  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:34.109076  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:34.109591  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:39.110515  2732 slave.cpp:944] Authentication timed out
>> W0719 12:24:39.111414  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:39.111649  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:39.111794  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:39.112321  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:44.112977  2726 slave.cpp:944] Authentication timed out
>> W0719 12:24:44.113622  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:44.113817  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:44.113931  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:44.114426  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:49.115818  2732 slave.cpp:944] Authentication timed out
>> W0719 12:24:49.116366  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:49.116511  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:49.116642  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:49.117146  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:54.117846  2731 slave.cpp:944] Authentication timed out
>> W0719 12:24:54.118671  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:54.118854  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:54.118928  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:54.119367  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:24:59.120332  2727 slave.cpp:944] Authentication timed out
>> W0719 12:24:59.120982  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:24:59.121171  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:24:59.121261  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:24:59.121898  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:04.123075  2728 slave.cpp:944] Authentication timed out
>> W0719 12:25:04.123872  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:04.124025  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:04.124083  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:04.124619  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:09.125983  2732 slave.cpp:944] Authentication timed out
>> W0719 12:25:09.126662  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:09.126938  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:09.127012  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:09.127579  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:14.128725  2725 slave.cpp:944] Authentication timed out
>> W0719 12:25:14.129214  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:14.129426  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:14.129490  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:14.129909  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:19.131160  2730 slave.cpp:944] Authentication timed out
>> W0719 12:25:19.131841  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:19.132031  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:19.132114  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:19.132634  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:24.133947  2729 slave.cpp:944] Authentication timed out
>> W0719 12:25:24.134681  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:24.134913  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:24.135006  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:24.135471  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:25:26.616324  2731 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169250278578171days
>> W0719 12:25:29.136682  2726 slave.cpp:944] Authentication timed out
>> W0719 12:25:29.137389  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:29.137576  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:29.137644  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:29.138147  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:34.139066  2729 slave.cpp:944] Authentication timed out
>> W0719 12:25:34.139564  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:34.139700  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:34.139741  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:34.140192  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:39.141010  2727 slave.cpp:944] Authentication timed out
>> W0719 12:25:39.141593  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:39.141824  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:39.141904  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:39.142398  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:44.143599  2725 slave.cpp:944] Authentication timed out
>> W0719 12:25:44.144395  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:44.144596  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:44.144675  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:44.145153  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:49.145772  2730 slave.cpp:944] Authentication timed out
>> W0719 12:25:49.146502  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:49.146764  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:49.146833  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:49.147356  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:54.147753  2726 slave.cpp:944] Authentication timed out
>> W0719 12:25:54.148535  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:54.148747  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:54.148828  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:54.149282  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:25:59.150144  2726 slave.cpp:944] Authentication timed out
>> W0719 12:25:59.150773  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:25:59.150938  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:25:59.151005  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:25:59.151433  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:04.152122  2729 slave.cpp:944] Authentication timed out
>> W0719 12:26:04.152866  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:04.153053  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:04.153146  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:04.153659  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:09.154232  2730 slave.cpp:944] Authentication timed out
>> W0719 12:26:09.154769  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:09.154965  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:09.155025  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:09.155586  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:14.156872  2731 slave.cpp:944] Authentication timed out
>> W0719 12:26:14.157555  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:14.157750  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:14.157811  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:14.158351  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:19.159608  2725 slave.cpp:944] Authentication timed out
>> W0719 12:26:19.160323  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:19.160508  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:19.160570  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:19.160997  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:24.161856  2731 slave.cpp:944] Authentication timed out
>> W0719 12:26:24.162442  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:24.162650  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:24.162756  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:24.163245  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:26:26.617436  2729 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169249836797558days
>> W0719 12:26:29.163816  2725 slave.cpp:944] Authentication timed out
>> W0719 12:26:29.164474  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:29.164662  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:29.164738  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:29.165218  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:34.165791  2728 slave.cpp:944] Authentication timed out
>> W0719 12:26:34.166488  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:34.166693  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:34.166795  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:34.167296  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:39.168243  2725 slave.cpp:944] Authentication timed out
>> W0719 12:26:39.168905  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:39.169137  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:39.169224  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:39.169710  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:44.170585  2729 slave.cpp:944] Authentication timed out
>> W0719 12:26:44.171337  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:44.171531  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:44.171602  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:44.172104  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:49.172936  2731 slave.cpp:944] Authentication timed out
>> W0719 12:26:49.173720  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:49.173892  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:49.173951  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:49.174450  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:54.175469  2729 slave.cpp:944] Authentication timed out
>> W0719 12:26:54.176153  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:54.176333  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:54.176393  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:54.176841  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:26:59.178102  2729 slave.cpp:944] Authentication timed out
>> W0719 12:26:59.178838  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:26:59.179049  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:26:59.179147  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:26:59.179589  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:04.180661  2726 slave.cpp:944] Authentication timed out
>> W0719 12:27:04.181210  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:04.181359  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:04.181416  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:04.181893  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:09.182955  2726 slave.cpp:944] Authentication timed out
>> W0719 12:27:09.183497  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:09.183751  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:09.183814  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:09.184252  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:14.184962  2730 slave.cpp:944] Authentication timed out
>> W0719 12:27:14.185637  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:14.185828  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:14.185912  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:14.186683  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:19.187254  2728 slave.cpp:944] Authentication timed out
>> W0719 12:27:19.188205  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:19.188354  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:19.188427  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:19.188961  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:24.189997  2728 slave.cpp:944] Authentication timed out
>> W0719 12:27:24.190727  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:24.190894  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:24.190964  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:24.191449  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:27:26.618481  2729 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169249615907257days
>> W0719 12:27:29.192710  2727 slave.cpp:944] Authentication timed out
>> W0719 12:27:29.193230  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:29.193442  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:29.193518  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:29.193967  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:34.195250  2728 slave.cpp:944] Authentication timed out
>> W0719 12:27:34.195855  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:34.196043  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:34.196128  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:34.196566  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:39.197839  2726 slave.cpp:944] Authentication timed out
>> W0719 12:27:39.198698  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:39.198894  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:39.198969  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:39.199517  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:44.200739  2726 slave.cpp:944] Authentication timed out
>> W0719 12:27:44.201248  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:44.201396  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:44.201454  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:44.201923  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:49.203218  2725 slave.cpp:944] Authentication timed out
>> W0719 12:27:49.203873  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:49.204198  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:49.204267  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:49.204649  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:54.205327  2732 slave.cpp:944] Authentication timed out
>> W0719 12:27:54.206069  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:54.206326  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:54.206396  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:54.206984  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:27:59.207499  2728 slave.cpp:944] Authentication timed out
>> W0719 12:27:59.208132  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:27:59.208325  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:27:59.208370  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:27:59.208850  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:04.210011  2731 slave.cpp:944] Authentication timed out
>> W0719 12:28:04.210692  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:04.210889  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:04.210955  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:04.211443  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:09.212529  2732 slave.cpp:944] Authentication timed out
>> W0719 12:28:09.213176  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:09.213325  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:09.213460  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:09.214041  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:14.215133  2732 slave.cpp:944] Authentication timed out
>> W0719 12:28:14.215914  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:14.216078  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:14.216132  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:14.216625  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:19.217176  2726 slave.cpp:944] Authentication timed out
>> W0719 12:28:19.217996  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:19.218209  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:19.218291  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:19.218855  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:24.220111  2732 slave.cpp:944] Authentication timed out
>> W0719 12:28:24.220759  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:24.220952  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:24.221051  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:24.221539  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:28:26.620170  2732 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169248732346030days
>> W0719 12:28:29.222877  2732 slave.cpp:944] Authentication timed out
>> W0719 12:28:29.223573  2732 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:29.223769  2732 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:29.223829  2732 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:29.224274  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:34.225024  2725 slave.cpp:944] Authentication timed out
>> W0719 12:28:34.225814  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:34.226063  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:34.226132  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:34.226645  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:39.227187  2726 slave.cpp:944] Authentication timed out
>> W0719 12:28:39.227819  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:39.227978  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:39.228101  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:39.228564  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:44.229616  2727 slave.cpp:944] Authentication timed out
>> W0719 12:28:44.230140  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:44.230283  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:44.230339  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:44.230813  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:49.231421  2731 slave.cpp:944] Authentication timed out
>> W0719 12:28:49.232055  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:49.232245  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:49.232305  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:49.232792  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:54.233731  2728 slave.cpp:944] Authentication timed out
>> W0719 12:28:54.234340  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:54.234493  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:54.234549  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:54.235028  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:28:59.235751  2728 slave.cpp:944] Authentication timed out
>> W0719 12:28:59.236279  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:28:59.236419  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:28:59.236475  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:28:59.236944  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:04.237779  2730 slave.cpp:944] Authentication timed out
>> W0719 12:29:04.238340  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:04.238487  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:04.238543  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:04.239001  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:09.240298  2728 slave.cpp:944] Authentication timed out
>> W0719 12:29:09.241166  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:09.241334  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:09.241394  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:09.241845  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:14.243237  2728 slave.cpp:944] Authentication timed out
>> W0719 12:29:14.244127  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:14.244290  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:14.244371  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:14.244820  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:19.245368  2726 slave.cpp:944] Authentication timed out
>> W0719 12:29:19.245990  2727 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:19.246197  2727 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:19.246279  2727 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:19.246924  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:24.248062  2725 slave.cpp:944] Authentication timed out
>> W0719 12:29:24.248940  2728 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:24.249105  2728 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:24.249163  2728 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:24.249614  2725 authenticatee.cpp:121] Creating new client
>> SASL connection
>> I0719 12:29:26.621341  2730 slave.cpp:4359] Current disk usage 1.87%. Max
>> allowed age: 6.169248842791180days
>> W0719 12:29:29.250006  2725 slave.cpp:944] Authentication timed out
>> W0719 12:29:29.250661  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:29.250864  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:29.250923  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:29.251415  2731 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:34.252017  2729 slave.cpp:944] Authentication timed out
>> W0719 12:29:34.252729  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:34.252897  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:34.252980  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:34.253417  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:39.254900  2725 slave.cpp:944] Authentication timed out
>> W0719 12:29:39.255656  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:39.255822  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:39.255878  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:39.256358  2729 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:44.257105  2731 slave.cpp:944] Authentication timed out
>> W0719 12:29:44.257777  2731 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:44.257964  2731 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:44.258023  2731 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:44.258486  2730 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:49.258904  2730 slave.cpp:944] Authentication timed out
>> W0719 12:29:49.259932  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:49.260124  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:49.260187  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:49.260723  2726 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:54.261531  2730 slave.cpp:944] Authentication timed out
>> W0719 12:29:54.262212  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:54.262387  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:54.262459  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:54.262948  2728 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:29:59.264312  2729 slave.cpp:944] Authentication timed out
>> W0719 12:29:59.264811  2729 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:29:59.264956  2729 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:29:59.265014  2729 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:29:59.265439  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:30:04.266129  2729 slave.cpp:944] Authentication timed out
>> W0719 12:30:04.266919  2725 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:30:04.267118  2725 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:30:04.267175  2725 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:30:04.267630  2727 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:30:09.268898  2730 slave.cpp:944] Authentication timed out
>> W0719 12:30:09.269462  2730 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:30:09.269618  2730 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:30:09.269675  2730 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:30:09.270092  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>> W0719 12:30:14.271193  2726 slave.cpp:944] Authentication timed out
>> W0719 12:30:14.271802  2726 slave.cpp:908] Failed to authenticate with
>> master master@10.5.7.4:5050: Authentication discarded
>> I0719 12:30:14.272011  2726 slave.cpp:859] Authenticating with master
>> master@10.5.7.4:5050
>> I0719 12:30:14.272073  2726 slave.cpp:864] Using default CRAM-MD5
>> authenticatee
>> I0719 12:30:14.272523  2732 authenticatee.cpp:121] Creating new client
>> SASL connection
>>
>>
>> On Mon, Jul 18, 2016 at 6:03 PM, Vinod Kone <vi...@apache.org> wrote:
>>
>>> Might be related to MESOS-2043
>>> <https://issues.apache.org/jira/browse/MESOS-2043>?
>>>
>>> Can you paste master and agent logs?
>>>
>>> On Mon, Jul 18, 2016 at 3:13 PM, Douglas Nelson <it...@gmail.com>
>>> wrote:
>>>
>>>> I have SSL enabled for mesos and for the most part everything seems to
>>>> be working fine. But when I stop a slave node for long enough that it shows
>>>> up with status LOST then I start up the slave again, registration with the
>>>> master fails:
>>>>
>>>> I0718 15:51:45.646260 16791 master.cpp:5495] Authenticating slave(1)@
>>>> 10.5.7.5:5051
>>>> I0718 15:51:45.646960 16791 authenticator.cpp:98] Creating new server
>>>> SASL connection
>>>> I0718 15:51:50.648329 16790 master.cpp:5481] Queuing up authentication
>>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>>> progress
>>>> W0718 15:51:50.648696 16790 master.cpp:5522] Failed to authenticate
>>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>>
>>>> It cycles through this over and over again until I restart the master
>>>> node. Is restarting the master the only way to handle re-authentication? I
>>>> expected it to be more automatic. Thanks!
>>>>
>>>
>>>
>>
>

Re: Possible authentication bug

Posted by Douglas Nelson <it...@gmail.com>.

I discovered something else with this issue today. Authentication only
fails every other time mesos attempts to register an agent node with the
master node. I've tried completely clearing out zookeeper and starting from
scratch with the same result. I deploy an application using marathon, kill
the agent, wait for it to unregister with the master, then start the agent
again. Repeating that process, every other time it's able to register. When
it isn't able to register I have to restart the master node before
registration completes. Could it be related to the app deployed by
marathon? Has anyone else seen this behavior?

If any other information is needed, let me know. I'm out of ideas about the
cause.

On Tue, Jul 19, 2016 at 12:47 PM, Douglas Nelson <it...@gmail.com>
wrote:

> MASTER:
>
> Log file created at: 2016/07/19 12:13:55
> Running on machine: dev1
> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
> I0719 12:13:55.437517 11025 logging.cpp:188] INFO level logging started!
> I0719 12:13:55.438292 11025 main.cpp:237] Build: 2016-07-08 16:57:43 by
> root
> I0719 12:13:55.438343 11025 main.cpp:239] Version: 0.28.2
> I0719 12:13:55.438967 11025 main.cpp:260] Using 'HierarchicalDRF' allocator
> I0719 12:13:55.449280 11025 leveldb.cpp:174] Opened db in 9.870872ms
> I0719 12:13:55.453482 11025 leveldb.cpp:181] Compacted db in 4.087025ms
> I0719 12:13:55.453719 11025 leveldb.cpp:196] Created db iterator in
> 132186ns
> I0719 12:13:55.453820 11025 leveldb.cpp:202] Seeked to beginning of db in
> 35181ns
> I0719 12:13:55.454460 11025 leveldb.cpp:271] Iterated through 3 keys in
> the db in 592652ns
> I0719 12:13:55.454795 11025 replica.cpp:779] Replica recovered with log
> positions 129 -> 130 with 0 holes and 0 unlearned
> I0719 12:13:55.461314 11049 log.cpp:236] Attempting to join replica to
> ZooKeeper group
> I0719 12:13:55.465665 11025 main.cpp:471] Starting Mesos master
> I0719 12:13:55.478101 11048 recover.cpp:447] Starting replica recovery
> I0719 12:13:55.482789 11025 master.cpp:375] Master
> c3c11abc-2f26-479f-8965-db34be5aa567 (dev1.something.com) started on
> 10.5.7.4:5050
> I0719 12:13:55.482998 11025 master.cpp:377] Flags at startup:
> --allocation_interval="1secs" --allocator="HierarchicalDRF"
> --authenticate="true" --authenticate_http="true"
> --authenticate_slaves="true" --authenticators="crammd5"
> --authorizers="local" --credentials="/etc/mesos/auth/credentials"
> --framework_sorter="drf" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --initialize_driver_logging="true"
> --log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
> --logging_level="INFO" --max_completed_frameworks="50"
> --max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
> --port="5050" --quiet="false" --quorum="1"
> --recovery_slave_removal_limit="100%" --registry="replicated_log"
> --registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
> --registry_strict="false" --root_submissions="true"
> --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
> --user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
> --work_dir="/var/lib/mesos" --zk="zk://10.5.7.4:2181/mesos"
> --zk_session_timeout="10secs"
> I0719 12:13:55.483695 11025 master.cpp:422] Master only allowing
> authenticated frameworks to register
> I0719 12:13:55.483753 11025 master.cpp:427] Master only allowing
> authenticated slaves to register
> I0719 12:13:55.483783 11025 credentials.hpp:35] Loading credentials for
> authentication from '/etc/mesos/auth/credentials'
> W0719 12:13:55.484066 11025 credentials.hpp:50] Permissions on credentials
> file '/etc/mesos/auth/credentials' are too open. It is recommended that
> your credentials file is NOT accessible by others.
> I0719 12:13:55.484253 11048 recover.cpp:473] Replica is in VOTING status
> I0719 12:13:55.484794 11048 recover.cpp:462] Recover process terminated
> I0719 12:13:55.494889 11025 master.cpp:467] Using default 'crammd5'
> authenticator
> I0719 12:13:55.495386 11025 authenticator.cpp:518] Initializing server SASL
> I0719 12:13:55.498042 11025 master.cpp:536] Using default 'basic' HTTP
> authenticator
> I0719 12:13:55.511752 11025 master.cpp:1650] Successfully attached file
> '/var/log/mesos/mesos-master.INFO'
> I0719 12:13:55.512787 11046 contender.cpp:147] Joining the ZK group
> I0719 12:13:55.603083 11046 group.cpp:349] Group process (group(1)@
> 10.5.7.4:5050) connected to ZooKeeper
> I0719 12:13:55.603395 11046 group.cpp:831] Syncing group operations: queue
> size (joins, cancels, datas) = (0, 0, 0)
> I0719 12:13:55.603469 11046 group.cpp:427] Trying to create path
> '/mesos/log_replicas' in ZooKeeper
> I0719 12:13:55.610569 11051 group.cpp:349] Group process (group(4)@
> 10.5.7.4:5050) connected to ZooKeeper
> I0719 12:13:55.610783 11051 group.cpp:831] Syncing group operations: queue
> size (joins, cancels, datas) = (0, 0, 0)
> I0719 12:13:55.610844 11051 group.cpp:427] Trying to create path '/mesos'
> in ZooKeeper
> I0719 12:13:55.610780 11044 group.cpp:349] Group process (group(3)@
> 10.5.7.4:5050) connected to ZooKeeper
> I0719 12:13:55.611119 11044 group.cpp:831] Syncing group operations: queue
> size (joins, cancels, datas) = (1, 0, 0)
> I0719 12:13:55.611163 11044 group.cpp:427] Trying to create path '/mesos'
> in ZooKeeper
> I0719 12:13:55.611829 11049 group.cpp:349] Group process (group(2)@
> 10.5.7.4:5050) connected to ZooKeeper
> I0719 12:13:55.611938 11049 group.cpp:831] Syncing group operations: queue
> size (joins, cancels, datas) = (1, 0, 0)
> I0719 12:13:55.611970 11049 group.cpp:427] Trying to create path
> '/mesos/log_replicas' in ZooKeeper
> I0719 12:13:55.652545 11048 contender.cpp:263] New candidate (id='2') has
> entered the contest for leadership
> I0719 12:13:55.656810 11046 network.hpp:413] ZooKeeper group memberships
> changed
> I0719 12:13:55.657207 11044 group.cpp:700] Trying to get
> '/mesos/log_replicas/0000000001' in ZooKeeper
> I0719 12:13:55.659225 11049 detector.cpp:152] Detected a new leader:
> (id='2')
> I0719 12:13:55.660096 11051 group.cpp:700] Trying to get
> '/mesos/json.info_0000000002' in ZooKeeper
> I0719 12:13:55.663472 11050 network.hpp:461] ZooKeeper group PIDs: {
> log-replica(1)@10.5.7.4:5050 }
> I0719 12:13:55.664608 11046 detector.cpp:479] A new leading master (UPID=
> master@10.5.7.4:5050) is detected
> I0719 12:13:55.665132 11049 master.cpp:1711] The newly elected leader is
> master@10.5.7.4:5050 with id c3c11abc-2f26-479f-8965-db34be5aa567
> I0719 12:13:55.665240 11049 master.cpp:1724] Elected as the leading master!
> I0719 12:13:55.665334 11049 master.cpp:1469] Recovering from registrar
> I0719 12:13:55.665786 11044 registrar.cpp:307] Recovering registrar
> I0719 12:13:55.668251 11049 log.cpp:659] Attempting to start the writer
> I0719 12:13:55.674710 11045 replica.cpp:493] Replica received implicit
> promise request from (6)@10.5.7.4:5050 with proposal 30
> I0719 12:13:55.676144 11045 leveldb.cpp:304] Persisting metadata (8 bytes)
> to leveldb took 1.29057ms
> I0719 12:13:55.676283 11045 replica.cpp:342] Persisted promised to 30
> I0719 12:13:55.678892 11044 coordinator.cpp:238] Coordinator attempting to
> fill missing positions
> I0719 12:13:55.679973 11047 log.cpp:675] Writer started with ending
> position 130
> I0719 12:13:55.684237 11051 leveldb.cpp:436] Reading position from leveldb
> took 180882ns
> I0719 12:13:55.684692 11051 leveldb.cpp:436] Reading position from leveldb
> took 86170ns
> I0719 12:13:55.689906 11045 registrar.cpp:340] Successfully fetched the
> registry (320B) in 23.838976ms
> I0719 12:13:55.690568 11045 registrar.cpp:439] Applied 1 operations in
> 253986ns; attempting to update the 'registry'
> I0719 12:13:55.695715 11051 log.cpp:683] Attempting to append 359 bytes to
> the log
> I0719 12:13:55.696043 11045 coordinator.cpp:348] Coordinator attempting to
> write APPEND action at position 131
> I0719 12:13:55.698156 11045 replica.cpp:537] Replica received write
> request for position 131 from (7)@10.5.7.4:5050
> I0719 12:13:55.699312 11045 leveldb.cpp:341] Persisting action (379 bytes)
> to leveldb took 918900ns
> I0719 12:13:55.699405 11045 replica.cpp:712] Persisted action at 131
> I0719 12:13:55.700897 11048 replica.cpp:691] Replica received learned
> notice for position 131 from @0.0.0.0:0
> I0719 12:13:55.701294 11048 leveldb.cpp:341] Persisting action (381 bytes)
> to leveldb took 305226ns
> I0719 12:13:55.701371 11048 replica.cpp:712] Persisted action at 131
> I0719 12:13:55.701427 11048 replica.cpp:697] Replica learned APPEND action
> at position 131
> I0719 12:13:55.705004 11051 registrar.cpp:484] Successfully updated the
> 'registry' in 14.150144ms
> I0719 12:13:55.705585 11051 registrar.cpp:370] Successfully recovered
> registrar
> I0719 12:13:55.705776 11047 log.cpp:702] Attempting to truncate the log to
> 131
> I0719 12:13:55.706045 11046 coordinator.cpp:348] Coordinator attempting to
> write TRUNCATE action at position 132
> I0719 12:13:55.707181 11044 replica.cpp:537] Replica received write
> request for position 132 from (8)@10.5.7.4:5050
> I0719 12:13:55.707342 11046 master.cpp:1521] Recovered 1 slaves from the
> Registry (320B) ; allowing 10mins for slaves to re-register
> I0719 12:13:55.708180 11044 leveldb.cpp:341] Persisting action (18 bytes)
> to leveldb took 889855ns
> I0719 12:13:55.708283 11044 replica.cpp:712] Persisted action at 132
> I0719 12:13:55.709445 11051 replica.cpp:691] Replica received learned
> notice for position 132 from @0.0.0.0:0
> I0719 12:13:55.709847 11051 leveldb.cpp:341] Persisting action (20 bytes)
> to leveldb took 300975ns
> I0719 12:13:55.709995 11051 leveldb.cpp:399] Deleting ~2 keys from leveldb
> took 69236ns
> I0719 12:13:55.710054 11051 replica.cpp:712] Persisted action at 132
> I0719 12:13:55.710115 11051 replica.cpp:697] Replica learned TRUNCATE
> action at position 132
> I0719 12:14:05.059528 11047 master.cpp:5495] Authenticating
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:05.062243 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:14:05.098544 11049 authenticator.cpp:203] Received SASL
> authentication start
> I0719 12:14:05.098788 11049 authenticator.cpp:325] Authentication requires
> more steps
> I0719 12:14:05.107225 11044 authenticator.cpp:231] Received SASL
> authentication step
> I0719 12:14:05.107561 11044 authenticator.cpp:317] Authentication success
> I0719 12:14:05.107964 11044 master.cpp:5525] Successfully authenticated
> principal 'dev' at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:05.110188 11046 master.cpp:2231] Received SUBSCRIBE call for
> framework 'marathon' at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:05.110599 11046 master.cpp:2302] Subscribing framework
> marathon with checkpointing enabled and capabilities [  ]
> I0719 12:14:05.112211 11051 hierarchical.cpp:265] Added framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:14:07.229689 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:14:07.250246 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:14:07.281565 11050 authenticator.cpp:203] Received SASL
> authentication start
> I0719 12:14:07.281793 11050 authenticator.cpp:325] Authentication requires
> more steps
> I0719 12:14:07.301153 11050 authenticator.cpp:231] Received SASL
> authentication step
> I0719 12:14:07.301486 11050 authenticator.cpp:317] Authentication success
> I0719 12:14:07.302227 11051 master.cpp:5525] Successfully authenticated
> principal 'dev' at slave(1)@10.5.7.5:5051
> I0719 12:14:07.307806 11051 master.cpp:4439] Re-registering slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com)
> I0719 12:14:07.310359 11044 registrar.cpp:439] Applied 1 operations in
> 176631ns; attempting to update the 'registry'
> I0719 12:14:07.316319 11049 log.cpp:683] Attempting to append 359 bytes to
> the log
> I0719 12:14:07.316714 11046 coordinator.cpp:348] Coordinator attempting to
> write APPEND action at position 133
> I0719 12:14:07.318164 11046 replica.cpp:537] Replica received write
> request for position 133 from (11)@10.5.7.4:5050
> I0719 12:14:07.318792 11046 leveldb.cpp:341] Persisting action (379 bytes)
> to leveldb took 519351ns
> I0719 12:14:07.318903 11046 replica.cpp:712] Persisted action at 133
> I0719 12:14:07.320116 11048 replica.cpp:691] Replica received learned
> notice for position 133 from @0.0.0.0:0
> I0719 12:14:07.320976 11048 leveldb.cpp:341] Persisting action (381 bytes)
> to leveldb took 785461ns
> I0719 12:14:07.321053 11048 replica.cpp:712] Persisted action at 133
> I0719 12:14:07.321106 11048 replica.cpp:697] Replica learned APPEND action
> at position 133
> I0719 12:14:07.323272 11046 registrar.cpp:484] Successfully updated the
> 'registry' in 12.685056ms
> I0719 12:14:07.323559 11048 log.cpp:702] Attempting to truncate the log to
> 133
> I0719 12:14:07.323832 11049 coordinator.cpp:348] Coordinator attempting to
> write TRUNCATE action at position 134
> I0719 12:14:07.324877 11049 replica.cpp:537] Replica received write
> request for position 134 from (12)@10.5.7.4:5050
> I0719 12:14:07.325728 11049 leveldb.cpp:341] Persisting action (18 bytes)
> to leveldb took 756749ns
> I0719 12:14:07.325804 11049 replica.cpp:712] Persisted action at 134
> W0719 12:14:07.326323 11047 master.cpp:6170] Possibly orphaned completed
> task webapp.42f99194-4dda-11e6-8041-02421568c935 of framework
> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001 that ran on slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com)
> I0719 12:14:07.326839 11049 replica.cpp:691] Replica received learned
> notice for position 134 from @0.0.0.0:0
> I0719 12:14:07.327742 11049 leveldb.cpp:341] Persisting action (20 bytes)
> to leveldb took 717423ns
> I0719 12:14:07.327713 11047 master.cpp:4521] Re-registered slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com) with cpus(*):4; mem(*):2929; disk(*):242457;
> ports(*):[31000-32000]
> I0719 12:14:07.328063 11047 master.cpp:4556] Sending updated checkpointed
> resources  to slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@
> 10.5.7.5:5051 (dev2.something.com)
> I0719 12:14:07.328157 11049 leveldb.cpp:399] Deleting ~2 keys from leveldb
> took 188898ns
> I0719 12:14:07.328233 11049 replica.cpp:712] Persisted action at 134
> I0719 12:14:07.328300 11049 replica.cpp:697] Replica learned TRUNCATE
> action at position 134
> I0719 12:14:07.329198 11046 hierarchical.cpp:473] Added slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) with
> cpus(*):4; mem(*):2929; disk(*):242457; ports(*):[31000-32000] (allocated: )
> I0719 12:14:07.330371 11044 master.cpp:4618] Received update of slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com) with total oversubscribed resources
> I0719 12:14:07.332751 11046 hierarchical.cpp:531] Slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) updated with
> oversubscribed resources  (total: cpus(*):4; mem(*):2929; disk(*):242457;
> ports(*):[31000-32000], allocated: cpus(*):4; mem(*):2929; disk(*):242457;
> ports(*):[31000-32000])
> I0719 12:14:07.332794 11047 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:07.432186 11045 master.cpp:3641] Processing DECLINE call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O0 ] for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:09.503839 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:14:09.877415 11044 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:09.877856 11044 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:14:09.880434 11051 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:09.886831 11048 master.cpp:3641] Processing DECLINE call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O1 ] for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:14.894800 11045 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:14.895149 11051 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:14:14.897150 11045 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:14.903162 11044 master.cpp:3641] Processing DECLINE call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O2 ] for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:19.714318 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:14:19.914870 11049 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:19.915210 11047 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:14:19.917094 11049 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:19.923250 11051 master.cpp:3641] Processing DECLINE call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O3 ] for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:20.019886 11050 master.cpp:4997] Performing implicit task
> state reconciliation for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:29.515525 11048 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:29.515820 11048 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:14:29.517138 11048 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:29.766815 11044 master.cpp:3104] Processing ACCEPT call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O4 ] on slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com) for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:29.771697 11049 master.hpp:177] Adding task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
> mem(*):32; ports(*):[31990-31990] on slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
> I0719 12:14:29.772017 11049 master.cpp:3589] Launching task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 with
> resources cpus(*):0.1; mem(*):32; ports(*):[31990-31990] on slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com)
> I0719 12:14:30.589148 11045 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:30.595021 11046 master.cpp:3641] Processing DECLINE call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O5 ] for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:14:30.610872 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:14:30.803971 11048 master.cpp:4763] Status update TASK_RUNNING
> (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 from slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com)
> I0719 12:14:30.804188 11048 master.cpp:4811] Forwarding status update
> TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:14:30.804988 11048 master.cpp:6421] Updating the state of task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_RUNNING,
> status update state: TASK_RUNNING)
> I0719 12:14:30.868975 11046 master.cpp:3918] Processing ACKNOWLEDGE call
> 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f for task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 on slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
> I0719 12:14:40.864478 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:14:51.114728 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:01.365391 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:08.869536 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51290 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:18.887311 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:28.904626 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:38.918462 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:48.932732 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:15:58.946310 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:16:08.961182 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:16:18.976491 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:16:29.148555 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:16:30.951886 11046 master.cpp:5324] Sending 1 offers to framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:16:30.958127 11047 master.cpp:3641] Processing DECLINE call for
> offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O6 ] for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:16:39.398538 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:16:49.648846 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:16:59.899797 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:17:07.350857 11047 master.cpp:242] Shutting down slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 due to health check timeout
> W0719 12:17:07.351191 11047 master.cpp:4930] Shutting down slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com) with message 'health check timed out'
> I0719 12:17:07.351920 11047 master.cpp:6201] Removing slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com): health check timed out
> I0719 12:17:07.352468 11047 master.cpp:6421] Updating the state of task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_LOST, status
> update state: TASK_LOST)
> I0719 12:17:07.352746 11045 hierarchical.cpp:505] Removed slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
> I0719 12:17:07.353837 11047 master.cpp:6487] Removing task
> webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
> mem(*):32; ports(*):[31990-31990] of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 on slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
> dev2.something.com)
> I0719 12:17:07.356307 11046 registrar.cpp:439] Applied 1 operations in
> 164323ns; attempting to update the 'registry'
> I0719 12:17:07.360155 11045 log.cpp:683] Attempting to append 184 bytes to
> the log
> I0719 12:17:07.360399 11050 coordinator.cpp:348] Coordinator attempting to
> write APPEND action at position 135
> I0719 12:17:07.361938 11051 replica.cpp:537] Replica received write
> request for position 135 from (40)@10.5.7.4:5050
> I0719 12:17:07.362778 11051 leveldb.cpp:341] Persisting action (204 bytes)
> to leveldb took 728456ns
> I0719 12:17:07.362862 11051 replica.cpp:712] Persisted action at 135
> I0719 12:17:07.363806 11051 replica.cpp:691] Replica received learned
> notice for position 135 from @0.0.0.0:0
> I0719 12:17:07.364334 11051 leveldb.cpp:341] Persisting action (206 bytes)
> to leveldb took 430978ns
> I0719 12:17:07.364414 11051 replica.cpp:712] Persisted action at 135
> I0719 12:17:07.364471 11051 replica.cpp:697] Replica learned APPEND action
> at position 135
> I0719 12:17:07.366155 11047 registrar.cpp:484] Successfully updated the
> 'registry' in 9.689088ms
> I0719 12:17:07.366540 11051 log.cpp:702] Attempting to truncate the log to
> 135
> I0719 12:17:07.366657 11046 master.cpp:6320] Removed slave
> ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com): health
> check timed out
> I0719 12:17:07.366739 11047 coordinator.cpp:348] Coordinator attempting to
> write TRUNCATE action at position 136
> I0719 12:17:07.366789 11046 master.cpp:4806] Sending status update
> TASK_LOST for task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 'Slave dev2.something.com
> removed: health check timed out'
> I0719 12:17:07.367236 11046 master.cpp:6343] Notifying framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 of lost
> slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) after
> recovering
> I0719 12:17:07.367993 11049 replica.cpp:537] Replica received write
> request for position 136 from (41)@10.5.7.4:5050
> I0719 12:17:07.368701 11049 leveldb.cpp:341] Persisting action (18 bytes)
> to leveldb took 607673ns
> I0719 12:17:07.368780 11049 replica.cpp:712] Persisted action at 136
> I0719 12:17:07.369793 11048 replica.cpp:691] Replica received learned
> notice for position 136 from @0.0.0.0:0
> I0719 12:17:07.374290 11048 leveldb.cpp:341] Persisting action (20 bytes)
> to leveldb took 4.333673ms
> I0719 12:17:07.374531 11048 leveldb.cpp:399] Deleting ~2 keys from leveldb
> took 142498ns
> I0719 12:17:07.374651 11048 replica.cpp:712] Persisted action at 136
> I0719 12:17:07.374763 11048 replica.cpp:697] Replica learned TRUNCATE
> action at position 136
> I0719 12:17:07.413800 11048 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:17:07.414114 11051 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:10.107131 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:17:12.435168 11051 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:17:12.435470 11051 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:17.455554 11049 master.cpp:3720] Processing REVIVE call for
> framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:17:17.456195 11044 hierarchical.cpp:988] Removed offer filters
> for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:20.620965 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:17:28.907829 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:28.908550 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:17:30.634392 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:17:33.887810 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:17:33.888640 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:17:33.888864 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:33.889627 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:17:38.890091 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:17:38.890705 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:17:38.890908 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:38.891654 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:17:41.619175 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:17:43.892462 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:17:43.893380 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:17:43.893582 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:43.894261 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:17:48.894907 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:17:48.895597 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:17:48.895742 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:48.896458 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:17:52.622318 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:17:53.897758 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:17:53.898336 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:17:53.898510 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:53.899430 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:17:58.900116 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:17:58.900677 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:17:58.901741 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:17:58.902381 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:03.621628 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:18:03.902536 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:03.903519 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:03.903714 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:03.904594 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:08.904714 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:18:08.905208 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:08.905773 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:08.905913 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:08.906513 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:13.907012 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:18:13.907600 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:13.907958 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:13.908123 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:13.908871 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:14.624475 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:18:18.909356 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:18.910174 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:18.910326 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:18.911255 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:23.911778 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:18:23.912201 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:23.912739 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:23.913128 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:23.913684 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:25.624198 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:18:28.914320 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:28.914964 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:28.915148 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:28.915825 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:33.916498 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:33.917145 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:33.917282 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:33.918520 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:36.625072 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:18:38.918712 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:38.919631 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:38.920052 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:38.920876 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:43.921413 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:43.922125 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:43.922269 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:43.923209 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:47.627984 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:18:48.923180 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:48.923815 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:48.924010 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:48.924589 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:53.925963 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:53.926895 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:53.927124 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:53.927942 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:18:58.626113 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:18:58.927940 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:18:58.928613 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:18:58.928766 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:18:58.930127 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:03.930527 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:03.931396 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:03.931637 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:03.932543 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:08.932302 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:19:08.932852 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:08.933034 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:08.933212 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:08.933835 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:09.627965 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:19:13.934926 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:13.935514 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:13.935757 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:13.936638 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:18.936506 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:19:18.937036 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:18.937420 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:18.937613 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:18.938341 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:20.624949 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:19:23.938432 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:23.939040 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:23.939214 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:23.940078 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:28.941543 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:28.942198 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:28.942435 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:28.943140 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:31.627547 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:19:33.943943 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:19:33.944599 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:33.944835 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:33.944988 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:33.945693 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:38.945986 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:19:38.946478 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:38.946812 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:38.946949 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:38.947438 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:42.628363 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:19:43.948642 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:43.949349 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:43.949550 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:43.950319 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:48.951385 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:48.951884 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:48.952000 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:48.952672 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:53.628947 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:19:53.954244 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:53.954866 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:53.955090 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:53.956058 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:19:58.956197 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:19:58.956797 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:19:58.957006 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:19:58.957918 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:03.958680 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:03.959282 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:03.959450 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:03.960024 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:04.630003 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:20:08.961760 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:20:08.962337 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:08.962745 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:08.962887 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:08.963418 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:13.964138 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:13.964747 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:13.965031 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:13.965772 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:15.630905 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:20:18.966778 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:18.967433 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:18.967687 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:18.968503 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:23.969029 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:23.969832 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:23.969976 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:23.970935 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:26.628504 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:20:28.971153 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:20:28.971578 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:28.971874 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:28.972044 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:28.972594 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:33.973855 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:20:33.974304 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:33.974576 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:33.974689 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:33.975183 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:37.629881 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:20:38.976158 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:38.977018 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:38.977200 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:38.978510 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:43.977927 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:43.978523 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:43.978688 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:43.979673 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:48.631947 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:20:48.980355 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:48.981098 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:48.981258 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:48.982522 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:53.983012 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:53.983757 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:53.983986 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:53.984787 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:58.985410 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:20:58.985873 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:20:58.986018 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:20:58.986199 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:20:58.986820 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:20:59.630745 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:21:03.987254 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:03.987884 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:03.988030 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:03.989013 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:08.989428 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:21:08.990020 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:08.990285 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:08.990468 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:08.990981 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:10.633591 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:21:13.992058 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:13.992918 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:13.993191 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:13.994236 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:18.994022 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:21:18.994477 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:18.994962 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:18.995141 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:18.995689 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:21.647516 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:21:23.996325 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:23.997014 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:23.997248 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:23.998145 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:28.998471 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:28.999652 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:28.999872 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:29.000628 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:32.635963 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:21:34.000746 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:34.001461 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:34.001718 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:34.002413 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:39.003231 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:39.003926 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:39.004137 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:39.004909 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:43.639329 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:21:44.005383 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:44.005898 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:44.006666 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:44.007246 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:49.007254 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:49.007946 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:49.008180 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:49.009217 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:54.009101 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:54.009840 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:54.010089 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:54.010777 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:21:54.634322 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:21:59.011060 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:21:59.011662 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:21:59.011852 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:21:59.012444 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:04.013264 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:22:04.013864 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:04.014137 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:04.014339 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:04.015051 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:05.633569 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:22:09.015625 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:09.016332 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:09.016549 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:09.017400 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:14.018102 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:14.018810 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:14.018975 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:14.020356 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:16.638113 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:22:19.020917 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:22:19.021430 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:19.021756 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:19.021895 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:19.022616 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:24.023571 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:24.024431 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:24.024754 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:24.026060 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:27.643009 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:22:29.026162 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:29.026885 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:29.027034 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:29.028298 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:34.028923 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:34.029616 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:34.029865 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:34.030767 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:38.638548 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:22:39.031050 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:39.031689 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:39.031838 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:39.032711 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:44.033393 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:44.034443 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:44.034685 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:44.035321 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:49.035828 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:49.036345 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:49.036553 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:49.037240 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:49.636526 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:22:54.038525 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:22:54.039108 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:54.039549 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:54.039691 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:54.040293 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:22:59.040702 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:22:59.041216 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:22:59.041524 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:22:59.041749 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:22:59.042407 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:00.639822 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:23:04.042881 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:04.043432 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:04.043550 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:04.044637 11045 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:23:09.044903 11051 master.cpp:5541] Authentication timed out
> W0719 12:23:09.045856 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:09.046164 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:09.046769 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:11.640506 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:23:14.047653 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:14.048249 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:14.048429 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:14.049170 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:19.049494 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:19.050199 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:19.050398 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:19.051105 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:22.639598 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:23:24.051370 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:24.052043 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:24.052255 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:24.053045 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:29.053254 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:23:29.053788 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:29.054085 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:29.054247 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:29.054867 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:33.639268 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:23:34.055438 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:23:34.055974 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:34.056213 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:34.056349 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:34.056926 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:39.057729 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:39.058291 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:39.058435 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:39.059099 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:44.059839 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:44.060503 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:44.060667 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:44.061758 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:44.639904 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:23:49.062311 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:49.062969 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:49.063107 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:49.063963 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:54.064702 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:54.065510 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:54.065732 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:54.066545 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:23:55.642850 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:23:59.067154 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:23:59.067770 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:23:59.068084 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:23:59.068683 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:04.069491 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:04.070219 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:04.070417 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:04.071239 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:06.643237 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:24:09.071045 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:09.071583 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:09.071717 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:09.072408 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:14.073580 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:14.074219 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:14.074439 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:14.075234 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:17.645912 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:24:19.076020 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:19.076668 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:19.077389 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:19.078135 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:20.012001 11051 master.cpp:5062] Performing explicit task
> state reconciliation for 1 tasks of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:24:20.012976 11051 master.cpp:4997] Performing implicit task
> state reconciliation for framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
> scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
> I0719 12:24:24.078232 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:24.078850 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:24.079015 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:24.079833 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:28.644161 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:24:29.080739 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:29.081439 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:29.081719 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:29.082607 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:34.083586 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:24:34.084177 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:34.084424 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:34.085206 11048 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:24:39.085710 11044 master.cpp:5541] Authentication timed out
> W0719 12:24:39.085863 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:39.086200 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:39.086688 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:39.645274 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:24:44.087625 11051 master.cpp:5541] Authentication timed out
> W0719 12:24:44.087796 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:44.088065 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:44.088768 11051 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:24:49.089455 11046 master.cpp:5541] Authentication timed out
> W0719 12:24:49.089862 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:49.090831 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:49.091661 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:24:50.645830 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:24:54.092033 11045 master.cpp:5541] Authentication timed out
> W0719 12:24:54.092311 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:54.093353 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:54.093953 11045 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:24:59.094514 11045 master.cpp:5541] Authentication timed out
> W0719 12:24:59.094877 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:24:59.095696 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:24:59.096318 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:01.646898 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:25:04.098826 11045 master.cpp:5541] Authentication timed out
> I0719 12:25:04.099218 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:25:04.099647 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:04.099804 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:04.100505 11048 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:25:09.101160 11048 master.cpp:5541] Authentication timed out
> W0719 12:25:09.101491 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:09.101846 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:09.102602 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:11.660732 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:25:14.102864 11046 master.cpp:5541] Authentication timed out
> W0719 12:25:14.103103 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:14.103487 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:14.104300 11051 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:25:19.104631 11046 master.cpp:5541] Authentication timed out
> W0719 12:25:19.104817 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:19.106343 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:19.106928 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:21.673857 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:25:24.107980 11050 master.cpp:5541] Authentication timed out
> W0719 12:25:24.108252 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:24.109048 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:24.109756 11050 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:25:29.110285 11046 master.cpp:5541] Authentication timed out
> W0719 12:25:29.110486 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:29.111874 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:29.112534 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:32.648926 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:25:34.113827 11046 master.cpp:5541] Authentication timed out
> W0719 12:25:34.114099 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:34.114449 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:34.115272 11046 authenticator.cpp:98] Creating new server SASL
> connection
> W0719 12:25:39.115715 11048 master.cpp:5541] Authentication timed out
> I0719 12:25:39.116320 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:25:39.116565 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:39.116698 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:39.117486 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:43.649667 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> W0719 12:25:44.117769 11050 master.cpp:5541] Authentication timed out
> W0719 12:25:44.118093 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:44.119288 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:44.119913 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:49.121246 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:25:49.122274 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:49.122517 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:49.123277 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:54.123177 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:25:54.123744 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:54.123908 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:54.124613 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:25:54.650367 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:25:59.125444 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:25:59.126139 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:25:59.126317 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:25:59.127210 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:04.127732 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:04.128376 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:04.128587 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:04.129299 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:05.650473 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:26:09.129295 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:09.129942 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:09.130143 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:09.130893 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:14.132257 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:14.132984 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:14.133221 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:14.133903 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:16.650934 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:26:19.134830 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:26:19.135447 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:19.135695 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:19.135893 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:19.136826 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:24.137058 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:26:24.137684 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:24.137899 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:24.138236 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:24.138895 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:27.653244 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:26:29.139055 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:26:29.139578 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:29.139806 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:29.139937 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:29.140700 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:34.141217 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:34.142405 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:34.142554 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:34.143790 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:38.652467 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:26:39.143749 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:39.144413 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:39.144654 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:39.145241 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:44.146155 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:44.146859 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:44.147155 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:44.147868 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:49.148578 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:49.149238 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:49.149387 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:49.150285 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:49.652683 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:26:54.150988 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:26:54.151628 11047 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:54.152030 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:54.152228 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:54.152897 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:26:59.153410 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:26:59.153934 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:26:59.154147 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:26:59.154831 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:00.653033 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:27:04.155833 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:04.156525 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:04.156755 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:04.157620 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:09.158335 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:09.158910 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:09.159360 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:09.159893 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:11.654698 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:27:14.160549 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:14.161084 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:14.161367 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:14.162240 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:19.162997 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:19.163769 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:19.163976 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:19.165292 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:22.657127 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:27:24.165325 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:27:24.165812 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:24.166124 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:24.166335 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:24.166985 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:29.167835 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:29.168339 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:29.168457 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:29.169409 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:33.654701 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:27:34.170714 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:34.171576 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:34.171741 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:34.172905 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:39.173723 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:39.174795 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:39.174945 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:39.175669 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:44.175698 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:44.176259 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:44.176416 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:44.177232 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:44.655261 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:27:49.178457 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:27:49.178927 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:49.179082 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:49.179266 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:49.179967 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:54.181052 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:54.181738 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:54.181941 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:54.182667 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:27:55.656306 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:27:59.182857 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:27:59.183506 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:27:59.183732 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:27:59.184350 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:04.185437 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:04.186125 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:04.186264 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:04.187189 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:06.656854 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:28:09.188138 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:09.188856 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:09.189314 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:09.190248 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:14.190521 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:14.191016 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:14.191254 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:14.191875 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:17.659039 11047 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:28:19.192914 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:28:19.193475 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:19.193802 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:19.193939 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:19.194689 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:24.195497 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:24.196179 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:24.196390 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:24.197120 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:28.658488 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:28:29.198254 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:28:29.198724 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:29.198997 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:29.199115 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:29.199853 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:34.200729 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:28:34.201292 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:34.201460 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:34.201683 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:34.202527 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:39.202571 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:39.203177 11045 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:39.203344 11045 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:39.204205 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:39.659113 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:28:44.204735 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:44.205322 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:44.205466 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:44.206336 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:49.206876 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:28:49.207523 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:49.208042 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:49.208204 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:49.208876 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:50.659682 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:28:54.208958 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:54.209754 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:54.209985 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:54.210769 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:28:59.211021 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:28:59.211598 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:28:59.211750 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:28:59.212461 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:01.659540 11046 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:29:04.212937 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:04.213676 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:04.213832 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:04.214392 11051 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:09.215848 11050 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:09.216425 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:09.216625 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:09.217329 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:12.663728 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:29:14.218930 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:14.219707 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:14.220160 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:14.220715 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:19.221014 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:19.221758 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:19.221928 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:19.222981 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:23.660816 11048 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:29:24.223538 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:24.224192 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:24.224408 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:24.225181 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:29.225445 11045 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:29.226102 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:29.226274 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:29.227026 11050 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:34.227371 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:34.227906 11050 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:34.228116 11050 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:34.228906 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:34.662215 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:29:39.230406 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:39.231156 11051 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:39.231384 11051 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:39.232285 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:44.232631 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:44.233209 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:44.233371 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:44.234177 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:45.662411 11051 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:29:49.234805 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:49.235468 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:49.235636 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:49.237004 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:54.236991 11049 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:54.237576 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:54.237751 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:54.238709 11045 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:29:56.663393 11045 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:29:59.239316 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:29:59.239837 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:29:59.240001 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:29:59.240638 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:04.241629 11044 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:04.242250 11044 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:04.242472 11044 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:04.243203 11044 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:07.677323 11050 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:30:09.244133 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:09.244830 11046 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:09.245038 11046 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:09.245756 11046 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:14.246692 11046 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:14.247503 11049 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:14.247711 11049 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:14.248601 11049 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:18.664516 11044 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:30:19.248931 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:19.249675 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:19.249899 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:19.250809 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:24.250947 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:30:24.251588 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:24.251802 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:24.252038 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:24.252806 11048 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:29.253676 11051 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:29.254382 11047 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:29.254613 11047 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:29.255362 11047 authenticator.cpp:98] Creating new server SASL
> connection
> I0719 12:30:29.665109 11049 http.cpp:312] HTTP GET for /master/state from
> 10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
> I0719 12:30:34.255038 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> I0719 12:30:34.255576 11048 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0719 12:30:34.255727 11048 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
> I0719 12:30:34.255898 11048 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0719 12:30:34.256418 11048 authenticator.cpp:98] Creating new server SASL
> connection
>
>
>
>
> AGENT:
>
> Log file created at: 2016/07/19 12:17:26
> Running on machine: dev2
> Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
> I0719 12:17:26.442286  2706 logging.cpp:188] INFO level logging started!
> I0719 12:17:26.442948  2706 main.cpp:223] Build: 2016-07-08 16:57:43 by
> root
> I0719 12:17:26.443025  2706 main.cpp:225] Version: 0.28.2
> I0719 12:17:26.563817  2706 containerizer.cpp:149] Using isolation:
> posix/cpu,posix/mem,filesystem/posix
> I0719 12:17:26.577617  2706 linux_launcher.cpp:101] Using
> /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
> I0719 12:17:26.587507  2706 main.cpp:328] Starting Mesos slave
> I0719 12:17:26.591217  2726 slave.cpp:193] Slave started on 1)@
> 10.5.7.5:5051
> I0719 12:17:26.591284  2726 slave.cpp:194] Flags at startup:
> --appc_simple_discovery_uri_prefix="http://"
> --appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5"
> --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos"
> --container_disk_watch_interval="15secs" --containerizers="docker,mesos"
> --credential="/etc/mesos/auth/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="
> https://registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker"
> --enforce_container_disk_quota="false"
> --executor_environment_variables="{"SSL_CERT_FILE":"\/etc\/mesos\/ssl\/mesos.pem","SSL_ENABLED":"true","SSL_KEY_FILE":"\/etc\/mesos\/ssl\/mesos.key"}"
> --executor_registration_timeout="5mins"
> --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos"
> --log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO"
> --master="zk://10.5.7.4:2181/mesos"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="1secs" --revocable_cpu_low_priority="true"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/var/lib/mesos"
> I0719 12:17:26.592231  2726 credentials.hpp:83] Loading credential for
> authentication from '/etc/mesos/auth/credential'
> W0719 12:17:26.592401  2726 credentials.hpp:98] Permissions on credential
> file '/etc/mesos/auth/credential' are too open. It is recommended that your
> credential file is NOT accessible by others.
> I0719 12:17:26.593298  2730 group.cpp:349] Group process (group(1)@
> 10.5.7.5:5051) connected to ZooKeeper
> I0719 12:17:26.593535  2730 group.cpp:831] Syncing group operations: queue
> size (joins, cancels, datas) = (0, 0, 0)
> I0719 12:17:26.593608  2730 group.cpp:427] Trying to create path '/mesos'
> in ZooKeeper
> I0719 12:17:26.600158  2725 detector.cpp:152] Detected a new leader:
> (id='2')
> I0719 12:17:26.600987  2727 group.cpp:700] Trying to get
> '/mesos/json.info_0000000002' in ZooKeeper
> I0719 12:17:26.603498  2726 slave.cpp:324] Slave using credential for: dev
> I0719 12:17:26.604095  2731 detector.cpp:479] A new leading master (UPID=
> master@10.5.7.4:5050) is detected
> I0719 12:17:26.605425  2726 slave.cpp:464] Slave resources: cpus(*):4;
> mem(*):2929; disk(*):242457; ports(*):[31000-32000]
> I0719 12:17:26.605607  2726 slave.cpp:472] Slave attributes: [  ]
> I0719 12:17:26.605659  2726 slave.cpp:477] Slave hostname:
> dev2.something.com
> I0719 12:17:26.612834  2729 state.cpp:58] Recovering state from
> '/var/lib/mesos/meta'
> I0719 12:17:26.630897  2731 slave.cpp:4638] Recovering framework
> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
> I0719 12:17:26.631350  2731 slave.cpp:5461] Recovering executor
> 'webapp.42f99194-4dda-11e6-8041-02421568c935' of framework
> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
> I0719 12:17:26.633368  2728 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935/runs/7b4fc655-5680-4581-8025-5c218de5e893'
> for gc 6.99456443088days in the future
> I0719 12:17:26.633872  2731 slave.cpp:4063] Cleaning up framework
> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
> I0719 12:17:26.633875  2728 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935/runs/7b4fc655-5680-4581-8025-5c218de5e893'
> for gc 6.99456442714074days in the future
> I0719 12:17:26.634140  2728 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935'
> for gc 6.99456442632days in the future
> I0719 12:17:26.634322  2731 slave.cpp:4638] Recovering framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:26.634459  2728 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935'
> for gc 6.99456442578667days in the future
> I0719 12:17:26.634485  2731 slave.cpp:5461] Recovering executor
> 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:26.634722  2728 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001'
> for gc 6.9999926607763days in the future
> I0719 12:17:26.634914  2728 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001'
> for gc 6.99999265988444days in the future
> I0719 12:17:26.634171  2729 status_update_manager.cpp:282] Closing status
> update streams for framework b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
> I0719 12:17:26.637401  2727 status_update_manager.cpp:200] Recovering
> status update manager
> I0719 12:17:26.637521  2727 status_update_manager.cpp:208] Recovering
> executor 'webapp.42f99194-4dda-11e6-8041-02421568c935' of framework
> b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
> I0719 12:17:26.637789  2727 status_update_manager.cpp:208] Recovering
> executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:26.642676  2726 docker.cpp:773] Recovering Docker containers
> I0719 12:17:26.643991  2731 containerizer.cpp:407] Recovering containerizer
> I0719 12:17:26.647322  2731 containerizer.cpp:456] Skipping recovery of
> executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 because it was not launched from
> mesos containerizer
> I0719 12:17:26.654330  2727 provisioner.cpp:245] Provisioner recovery
> complete
> I0719 12:17:26.877194  2726 docker.cpp:877] Recovering container
> '65fa6566-806b-407f-a273-ef3b94f6dede' for executor
> 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
> 'ee03b9d8-797a-421b-9542-652e53fbc4fb-0000'
> I0719 12:17:26.883574  2727 slave.cpp:4490] Sending reconnect request to
> executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 at executor(1)@10.5.7.5:51307
> I0719 12:17:26.900796  2728 slave.cpp:2799] Re-registering executor
> 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
> ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
> I0719 12:17:26.993151  2729 docker.cpp:1427] Updated 'cpu.shares' to 204
> at
> /sys/fs/cgroup/cpu/docker/9efb4b79f889d69111e33dbce67bd62f8f82f3068d76a0b53118040f00f1ea5e
> for container 65fa6566-806b-407f-a273-ef3b94f6dede
> I0719 12:17:26.994645  2729 docker.cpp:1462] Updated
> 'memory.soft_limit_in_bytes' to 64MB for container
> 65fa6566-806b-407f-a273-ef3b94f6dede
> I0719 12:17:26.997007  2729 docker.cpp:1488] Updated
> 'memory.limit_in_bytes' to 64MB at
> /sys/fs/cgroup/memory/docker/9efb4b79f889d69111e33dbce67bd62f8f82f3068d76a0b53118040f00f1ea5e
> for container 65fa6566-806b-407f-a273-ef3b94f6dede
> I0719 12:17:28.886029  2725 slave.cpp:2952] Cleaning up un-reregistered
> executors
> I0719 12:17:28.886433  2725 slave.cpp:4550] Finished recovery
> I0719 12:17:28.887312  2725 slave.cpp:4583] Garbage collecting old slave
> 0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0
> I0719 12:17:28.887639  2725 slave.cpp:4583] Garbage collecting old slave
> 6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0
> I0719 12:17:28.887648  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0' for gc
> 6.99998972913185days in the future
> I0719 12:17:28.887792  2725 slave.cpp:4583] Garbage collecting old slave
> 31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1
> I0719 12:17:28.887933  2725 slave.cpp:4583] Garbage collecting old slave
> 9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0
> I0719 12:17:28.887967  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0' for gc
> 6.99998972714074days in the future
> I0719 12:17:28.888082  2725 slave.cpp:4583] Garbage collecting old slave
> ca647578-5569-4990-bce2-2bf86ffacde3-S0
> I0719 12:17:28.888191  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0' for gc
> 6.99998972571852days in the future
> I0719 12:17:28.888355  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0' for gc
> 6.99998972497778days in the future
> I0719 12:17:28.888499  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1' for gc
> 6.99998972407407days in the future
> I0719 12:17:28.888689  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1' for gc
> 6.99998972341333days in the future
> I0719 12:17:28.888829  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0' for gc
> 6.99998972240593days in the future
> I0719 12:17:28.888967  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0' for gc
> 6.99998972164444days in the future
> I0719 12:17:28.889101  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/slaves/ca647578-5569-4990-bce2-2bf86ffacde3-S0' for gc
> 6.99998972040593days in the future
> I0719 12:17:28.889317  2732 gc.cpp:55] Scheduling
> '/var/lib/mesos/meta/slaves/ca647578-5569-4990-bce2-2bf86ffacde3-S0' for gc
> 6.99998971916741days in the future
> I0719 12:17:28.889744  2732 status_update_manager.cpp:174] Pausing sending
> status updates
> I0719 12:17:28.889770  2725 slave.cpp:796] New master detected at
> master@10.5.7.4:5050
> I0719 12:17:28.909875  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:28.909971  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:28.910511  2725 slave.cpp:832] Detecting new master
> I0719 12:17:28.911711  2732 authenticatee.cpp:97] Initializing client SASL
> I0719 12:17:28.914371  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:17:33.911911  2731 slave.cpp:944] Authentication timed out
> W0719 12:17:33.912632  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:17:33.912788  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:33.912845  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:33.913310  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:17:38.914300  2730 slave.cpp:944] Authentication timed out
> W0719 12:17:38.915067  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:17:38.915307  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:38.915366  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:38.915830  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:17:43.916915  2725 slave.cpp:944] Authentication timed out
> W0719 12:17:43.917677  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:17:43.917862  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:43.917974  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:43.918507  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:17:48.918875  2730 slave.cpp:944] Authentication timed out
> W0719 12:17:48.919649  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:17:48.919847  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:48.919919  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:48.920433  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:17:53.921869  2727 slave.cpp:944] Authentication timed out
> W0719 12:17:53.922817  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:17:53.922976  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:53.923032  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:53.923563  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:17:58.924680  2732 slave.cpp:944] Authentication timed out
> W0719 12:17:58.925246  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:17:58.925434  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:17:58.925498  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:17:58.926103  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:03.927134  2730 slave.cpp:944] Authentication timed out
> W0719 12:18:03.927891  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:03.928052  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:03.928112  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:03.928536  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:08.929105  2728 slave.cpp:944] Authentication timed out
> W0719 12:18:08.929832  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:08.930012  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:08.930093  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:08.930577  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:13.931543  2729 slave.cpp:944] Authentication timed out
> W0719 12:18:13.932309  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:13.932477  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:13.932550  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:13.933042  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:18.933753  2731 slave.cpp:944] Authentication timed out
> W0719 12:18:18.934530  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:18.934729  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:18.934792  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:18.935309  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:23.936511  2729 slave.cpp:944] Authentication timed out
> W0719 12:18:23.937216  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:23.937366  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:23.937425  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:23.937896  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:18:26.607225  2729 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169253812823067days
> W0719 12:18:28.938818  2726 slave.cpp:944] Authentication timed out
> W0719 12:18:28.939522  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:28.939715  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:28.939811  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:28.940253  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:33.941254  2725 slave.cpp:944] Authentication timed out
> W0719 12:18:33.941874  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:33.942029  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:33.942085  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:33.942587  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:38.943189  2728 slave.cpp:944] Authentication timed out
> W0719 12:18:38.943855  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:38.944048  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:38.944110  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:38.944572  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:43.945793  2725 slave.cpp:944] Authentication timed out
> W0719 12:18:43.946396  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:43.946601  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:43.946729  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:43.947180  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:48.948164  2731 slave.cpp:944] Authentication timed out
> W0719 12:18:48.948698  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:48.948873  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:48.948927  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:48.949345  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:53.950121  2727 slave.cpp:944] Authentication timed out
> W0719 12:18:53.950748  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:53.951323  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:53.951418  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:53.951922  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:18:58.952540  2727 slave.cpp:944] Authentication timed out
> W0719 12:18:58.953218  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:18:58.953387  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:18:58.953461  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:18:58.953919  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:03.954779  2729 slave.cpp:944] Authentication timed out
> W0719 12:19:03.955401  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:03.955580  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:03.955657  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:03.956127  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:08.957002  2727 slave.cpp:944] Authentication timed out
> W0719 12:19:08.957651  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:08.957834  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:08.957892  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:08.958351  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:13.959039  2731 slave.cpp:944] Authentication timed out
> W0719 12:19:13.959734  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:13.959925  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:13.959991  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:13.960530  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:18.961040  2730 slave.cpp:944] Authentication timed out
> W0719 12:19:18.961683  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:18.961946  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:18.962019  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:18.962450  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:23.963018  2725 slave.cpp:944] Authentication timed out
> W0719 12:19:23.963678  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:23.963838  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:23.963894  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:23.964344  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:19:26.608978  2729 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169253371042454days
> W0719 12:19:28.965499  2726 slave.cpp:944] Authentication timed out
> W0719 12:19:28.966660  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:28.966851  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:28.966958  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:28.967434  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:33.968541  2727 slave.cpp:944] Authentication timed out
> W0719 12:19:33.969198  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:33.969369  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:33.969426  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:33.969883  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:38.970623  2732 slave.cpp:944] Authentication timed out
> W0719 12:19:38.971215  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:38.971395  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:38.971454  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:38.971917  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:43.973112  2731 slave.cpp:944] Authentication timed out
> W0719 12:19:43.973659  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:43.973954  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:43.974014  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:43.974584  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:48.975668  2731 slave.cpp:944] Authentication timed out
> W0719 12:19:48.976529  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:48.976691  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:48.976758  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:48.977280  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:53.978732  2725 slave.cpp:944] Authentication timed out
> W0719 12:19:53.979405  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:53.979600  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:53.979676  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:53.980168  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:19:58.980532  2726 slave.cpp:944] Authentication timed out
> W0719 12:19:58.981240  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:19:58.981444  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:19:58.981531  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:19:58.982074  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:03.983041  2732 slave.cpp:944] Authentication timed out
> W0719 12:20:03.983832  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:03.984200  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:03.984313  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:03.984802  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:08.986167  2726 slave.cpp:944] Authentication timed out
> W0719 12:20:08.986836  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:08.986984  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:08.987040  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:08.987558  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:13.988708  2731 slave.cpp:944] Authentication timed out
> W0719 12:20:13.989320  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:13.989475  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:13.989531  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:13.989976  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:18.991140  2729 slave.cpp:944] Authentication timed out
> W0719 12:20:18.991741  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:18.991935  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:18.992003  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:18.992486  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:23.993616  2732 slave.cpp:944] Authentication timed out
> W0719 12:20:23.994303  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:23.994451  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:23.994508  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:23.994971  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:20:26.610416  2725 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169252929261840days
> W0719 12:20:28.995944  2725 slave.cpp:944] Authentication timed out
> W0719 12:20:28.996537  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:28.996763  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:28.996839  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:28.997261  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:33.998452  2726 slave.cpp:944] Authentication timed out
> W0719 12:20:33.999094  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:33.999261  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:33.999356  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:33.999822  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:39.000432  2726 slave.cpp:944] Authentication timed out
> W0719 12:20:39.001195  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:39.001364  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:39.001435  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:39.001938  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:44.002457  2732 slave.cpp:944] Authentication timed out
> W0719 12:20:44.003163  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:44.003326  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:44.003382  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:44.003878  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:49.004884  2731 slave.cpp:944] Authentication timed out
> W0719 12:20:49.005506  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:49.005669  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:49.005765  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:49.006237  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:54.007592  2726 slave.cpp:944] Authentication timed out
> W0719 12:20:54.008363  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:54.008564  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:54.008625  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:54.009052  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:20:59.009913  2727 slave.cpp:944] Authentication timed out
> W0719 12:20:59.010506  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:20:59.010984  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:20:59.011035  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:20:59.011549  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:04.012027  2725 slave.cpp:944] Authentication timed out
> W0719 12:21:04.012603  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:04.012756  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:04.012812  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:04.013241  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:09.013691  2727 slave.cpp:944] Authentication timed out
> W0719 12:21:09.014644  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:09.014842  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:09.014914  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:09.015367  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:14.016613  2727 slave.cpp:944] Authentication timed out
> W0719 12:21:14.017313  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:14.017554  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:14.017611  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:14.018029  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:19.018718  2731 slave.cpp:944] Authentication timed out
> W0719 12:21:19.019327  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:19.019505  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:19.019610  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:19.020059  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:24.020687  2729 slave.cpp:944] Authentication timed out
> W0719 12:21:24.021451  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:24.021648  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:24.021708  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:24.022183  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:21:26.611302  2729 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169252045700625days
> W0719 12:21:29.023134  2732 slave.cpp:944] Authentication timed out
> W0719 12:21:29.023824  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:29.023972  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:29.024029  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:29.024477  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:34.024953  2730 slave.cpp:944] Authentication timed out
> W0719 12:21:34.025951  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:34.026146  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:34.026221  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:34.026741  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:39.027717  2731 slave.cpp:944] Authentication timed out
> W0719 12:21:39.028527  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:39.028733  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:39.028796  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:39.029225  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:44.029718  2732 slave.cpp:944] Authentication timed out
> W0719 12:21:44.030495  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:44.030712  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:44.030787  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:44.031285  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:49.031599  2726 slave.cpp:944] Authentication timed out
> W0719 12:21:49.032201  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:49.032347  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:49.032402  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:49.033093  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:54.033625  2732 slave.cpp:944] Authentication timed out
> W0719 12:21:54.034369  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:54.034518  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:54.034577  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:54.035073  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:21:59.035948  2729 slave.cpp:944] Authentication timed out
> W0719 12:21:59.036551  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:21:59.036703  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:21:59.036761  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:21:59.037250  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:04.037871  2729 slave.cpp:944] Authentication timed out
> W0719 12:22:04.038518  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:04.038727  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:04.038805  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:04.039253  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:09.040071  2726 slave.cpp:944] Authentication timed out
> W0719 12:22:09.040676  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:09.040865  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:09.040998  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:09.041527  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:14.042394  2731 slave.cpp:944] Authentication timed out
> W0719 12:22:14.043166  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:14.043345  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:14.043403  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:14.043896  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:19.045148  2731 slave.cpp:944] Authentication timed out
> W0719 12:22:19.045684  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:19.046054  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:19.046166  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:19.046763  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:24.048012  2730 slave.cpp:944] Authentication timed out
> W0719 12:22:24.048894  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:24.049060  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:24.049118  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:24.049540  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:22:26.612486  2730 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169251935255463days
> W0719 12:22:29.050376  2732 slave.cpp:944] Authentication timed out
> W0719 12:22:29.051234  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:29.051421  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:29.051477  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:29.051931  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:34.053143  2730 slave.cpp:944] Authentication timed out
> W0719 12:22:34.053758  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:34.054283  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:34.054349  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:34.054812  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:39.055835  2725 slave.cpp:944] Authentication timed out
> W0719 12:22:39.056479  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:39.056671  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:39.056728  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:39.057149  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:44.057876  2727 slave.cpp:944] Authentication timed out
> W0719 12:22:44.058470  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:44.058732  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:44.058861  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:44.059375  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:49.060206  2728 slave.cpp:944] Authentication timed out
> W0719 12:22:49.061079  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:49.061347  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:49.061414  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:49.061935  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:54.062958  2729 slave.cpp:944] Authentication timed out
> W0719 12:22:54.063654  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:54.063827  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:54.063910  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:54.064379  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:22:59.065388  2726 slave.cpp:944] Authentication timed out
> W0719 12:22:59.065942  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:22:59.066133  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:22:59.066200  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:22:59.066651  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:04.067623  2726 slave.cpp:944] Authentication timed out
> W0719 12:23:04.068518  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:04.068675  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:04.068732  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:04.069146  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:09.069924  2728 slave.cpp:944] Authentication timed out
> W0719 12:23:09.070524  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:09.071009  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:09.071141  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:09.071633  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:14.072458  2726 slave.cpp:944] Authentication timed out
> W0719 12:23:14.073009  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:14.073253  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:14.073353  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:14.073799  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:19.074239  2730 slave.cpp:944] Authentication timed out
> W0719 12:23:19.074939  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:19.075140  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:19.075202  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:19.075644  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:24.076045  2731 slave.cpp:944] Authentication timed out
> W0719 12:23:24.076644  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:24.076799  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:24.076855  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:24.077338  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:23:26.613450  2731 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169251714365162days
> W0719 12:23:29.077744  2732 slave.cpp:944] Authentication timed out
> W0719 12:23:29.078488  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:29.078699  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:29.078765  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:29.079268  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:34.079994  2726 slave.cpp:944] Authentication timed out
> W0719 12:23:34.080622  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:34.080775  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:34.080831  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:34.081279  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:39.082494  2732 slave.cpp:944] Authentication timed out
> W0719 12:23:39.083204  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:39.083395  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:39.083456  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:39.083904  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:44.084430  2728 slave.cpp:944] Authentication timed out
> W0719 12:23:44.085264  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:44.085417  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:44.085474  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:44.085904  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:49.086913  2732 slave.cpp:944] Authentication timed out
> W0719 12:23:49.087604  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:49.087771  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:49.087828  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:49.088310  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:54.089237  2730 slave.cpp:944] Authentication timed out
> W0719 12:23:54.089737  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:54.089936  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:54.090013  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:54.090585  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:23:59.091807  2729 slave.cpp:944] Authentication timed out
> W0719 12:23:59.092536  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:23:59.092717  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:23:59.092798  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:23:59.093231  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:04.094005  2730 slave.cpp:944] Authentication timed out
> W0719 12:24:04.094712  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:04.094916  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:04.094985  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:04.095445  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:09.095959  2731 slave.cpp:944] Authentication timed out
> W0719 12:24:09.096580  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:09.096776  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:09.096835  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:09.097257  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:14.098148  2727 slave.cpp:944] Authentication timed out
> W0719 12:24:14.098824  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:14.099046  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:14.099112  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:14.099601  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:19.100690  2725 slave.cpp:944] Authentication timed out
> W0719 12:24:19.101272  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:19.101482  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:19.101557  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:19.102108  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:24.102840  2731 slave.cpp:944] Authentication timed out
> W0719 12:24:24.103386  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:24.103564  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:24.103623  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:24.104046  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:24:26.614562  2729 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169251272584549days
> W0719 12:24:29.105397  2726 slave.cpp:944] Authentication timed out
> W0719 12:24:29.105924  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:29.106081  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:29.106142  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:29.106680  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:34.108180  2727 slave.cpp:944] Authentication timed out
> W0719 12:24:34.108861  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:34.109020  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:34.109076  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:34.109591  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:39.110515  2732 slave.cpp:944] Authentication timed out
> W0719 12:24:39.111414  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:39.111649  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:39.111794  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:39.112321  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:44.112977  2726 slave.cpp:944] Authentication timed out
> W0719 12:24:44.113622  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:44.113817  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:44.113931  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:44.114426  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:49.115818  2732 slave.cpp:944] Authentication timed out
> W0719 12:24:49.116366  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:49.116511  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:49.116642  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:49.117146  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:54.117846  2731 slave.cpp:944] Authentication timed out
> W0719 12:24:54.118671  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:54.118854  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:54.118928  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:54.119367  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:24:59.120332  2727 slave.cpp:944] Authentication timed out
> W0719 12:24:59.120982  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:24:59.121171  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:24:59.121261  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:24:59.121898  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:04.123075  2728 slave.cpp:944] Authentication timed out
> W0719 12:25:04.123872  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:04.124025  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:04.124083  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:04.124619  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:09.125983  2732 slave.cpp:944] Authentication timed out
> W0719 12:25:09.126662  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:09.126938  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:09.127012  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:09.127579  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:14.128725  2725 slave.cpp:944] Authentication timed out
> W0719 12:25:14.129214  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:14.129426  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:14.129490  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:14.129909  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:19.131160  2730 slave.cpp:944] Authentication timed out
> W0719 12:25:19.131841  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:19.132031  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:19.132114  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:19.132634  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:24.133947  2729 slave.cpp:944] Authentication timed out
> W0719 12:25:24.134681  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:24.134913  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:24.135006  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:24.135471  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:25:26.616324  2731 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169250278578171days
> W0719 12:25:29.136682  2726 slave.cpp:944] Authentication timed out
> W0719 12:25:29.137389  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:29.137576  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:29.137644  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:29.138147  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:34.139066  2729 slave.cpp:944] Authentication timed out
> W0719 12:25:34.139564  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:34.139700  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:34.139741  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:34.140192  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:39.141010  2727 slave.cpp:944] Authentication timed out
> W0719 12:25:39.141593  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:39.141824  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:39.141904  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:39.142398  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:44.143599  2725 slave.cpp:944] Authentication timed out
> W0719 12:25:44.144395  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:44.144596  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:44.144675  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:44.145153  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:49.145772  2730 slave.cpp:944] Authentication timed out
> W0719 12:25:49.146502  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:49.146764  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:49.146833  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:49.147356  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:54.147753  2726 slave.cpp:944] Authentication timed out
> W0719 12:25:54.148535  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:54.148747  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:54.148828  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:54.149282  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:25:59.150144  2726 slave.cpp:944] Authentication timed out
> W0719 12:25:59.150773  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:25:59.150938  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:25:59.151005  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:25:59.151433  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:04.152122  2729 slave.cpp:944] Authentication timed out
> W0719 12:26:04.152866  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:04.153053  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:04.153146  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:04.153659  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:09.154232  2730 slave.cpp:944] Authentication timed out
> W0719 12:26:09.154769  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:09.154965  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:09.155025  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:09.155586  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:14.156872  2731 slave.cpp:944] Authentication timed out
> W0719 12:26:14.157555  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:14.157750  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:14.157811  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:14.158351  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:19.159608  2725 slave.cpp:944] Authentication timed out
> W0719 12:26:19.160323  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:19.160508  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:19.160570  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:19.160997  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:24.161856  2731 slave.cpp:944] Authentication timed out
> W0719 12:26:24.162442  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:24.162650  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:24.162756  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:24.163245  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:26:26.617436  2729 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169249836797558days
> W0719 12:26:29.163816  2725 slave.cpp:944] Authentication timed out
> W0719 12:26:29.164474  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:29.164662  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:29.164738  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:29.165218  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:34.165791  2728 slave.cpp:944] Authentication timed out
> W0719 12:26:34.166488  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:34.166693  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:34.166795  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:34.167296  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:39.168243  2725 slave.cpp:944] Authentication timed out
> W0719 12:26:39.168905  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:39.169137  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:39.169224  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:39.169710  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:44.170585  2729 slave.cpp:944] Authentication timed out
> W0719 12:26:44.171337  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:44.171531  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:44.171602  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:44.172104  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:49.172936  2731 slave.cpp:944] Authentication timed out
> W0719 12:26:49.173720  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:49.173892  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:49.173951  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:49.174450  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:54.175469  2729 slave.cpp:944] Authentication timed out
> W0719 12:26:54.176153  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:54.176333  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:54.176393  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:54.176841  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:26:59.178102  2729 slave.cpp:944] Authentication timed out
> W0719 12:26:59.178838  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:26:59.179049  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:26:59.179147  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:26:59.179589  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:04.180661  2726 slave.cpp:944] Authentication timed out
> W0719 12:27:04.181210  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:04.181359  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:04.181416  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:04.181893  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:09.182955  2726 slave.cpp:944] Authentication timed out
> W0719 12:27:09.183497  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:09.183751  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:09.183814  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:09.184252  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:14.184962  2730 slave.cpp:944] Authentication timed out
> W0719 12:27:14.185637  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:14.185828  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:14.185912  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:14.186683  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:19.187254  2728 slave.cpp:944] Authentication timed out
> W0719 12:27:19.188205  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:19.188354  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:19.188427  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:19.188961  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:24.189997  2728 slave.cpp:944] Authentication timed out
> W0719 12:27:24.190727  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:24.190894  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:24.190964  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:24.191449  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:27:26.618481  2729 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169249615907257days
> W0719 12:27:29.192710  2727 slave.cpp:944] Authentication timed out
> W0719 12:27:29.193230  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:29.193442  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:29.193518  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:29.193967  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:34.195250  2728 slave.cpp:944] Authentication timed out
> W0719 12:27:34.195855  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:34.196043  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:34.196128  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:34.196566  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:39.197839  2726 slave.cpp:944] Authentication timed out
> W0719 12:27:39.198698  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:39.198894  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:39.198969  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:39.199517  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:44.200739  2726 slave.cpp:944] Authentication timed out
> W0719 12:27:44.201248  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:44.201396  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:44.201454  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:44.201923  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:49.203218  2725 slave.cpp:944] Authentication timed out
> W0719 12:27:49.203873  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:49.204198  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:49.204267  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:49.204649  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:54.205327  2732 slave.cpp:944] Authentication timed out
> W0719 12:27:54.206069  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:54.206326  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:54.206396  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:54.206984  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:27:59.207499  2728 slave.cpp:944] Authentication timed out
> W0719 12:27:59.208132  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:27:59.208325  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:27:59.208370  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:27:59.208850  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:04.210011  2731 slave.cpp:944] Authentication timed out
> W0719 12:28:04.210692  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:04.210889  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:04.210955  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:04.211443  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:09.212529  2732 slave.cpp:944] Authentication timed out
> W0719 12:28:09.213176  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:09.213325  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:09.213460  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:09.214041  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:14.215133  2732 slave.cpp:944] Authentication timed out
> W0719 12:28:14.215914  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:14.216078  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:14.216132  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:14.216625  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:19.217176  2726 slave.cpp:944] Authentication timed out
> W0719 12:28:19.217996  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:19.218209  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:19.218291  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:19.218855  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:24.220111  2732 slave.cpp:944] Authentication timed out
> W0719 12:28:24.220759  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:24.220952  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:24.221051  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:24.221539  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:28:26.620170  2732 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169248732346030days
> W0719 12:28:29.222877  2732 slave.cpp:944] Authentication timed out
> W0719 12:28:29.223573  2732 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:29.223769  2732 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:29.223829  2732 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:29.224274  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:34.225024  2725 slave.cpp:944] Authentication timed out
> W0719 12:28:34.225814  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:34.226063  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:34.226132  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:34.226645  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:39.227187  2726 slave.cpp:944] Authentication timed out
> W0719 12:28:39.227819  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:39.227978  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:39.228101  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:39.228564  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:44.229616  2727 slave.cpp:944] Authentication timed out
> W0719 12:28:44.230140  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:44.230283  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:44.230339  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:44.230813  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:49.231421  2731 slave.cpp:944] Authentication timed out
> W0719 12:28:49.232055  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:49.232245  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:49.232305  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:49.232792  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:54.233731  2728 slave.cpp:944] Authentication timed out
> W0719 12:28:54.234340  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:54.234493  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:54.234549  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:54.235028  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:28:59.235751  2728 slave.cpp:944] Authentication timed out
> W0719 12:28:59.236279  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:28:59.236419  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:28:59.236475  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:28:59.236944  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:04.237779  2730 slave.cpp:944] Authentication timed out
> W0719 12:29:04.238340  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:04.238487  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:04.238543  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:04.239001  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:09.240298  2728 slave.cpp:944] Authentication timed out
> W0719 12:29:09.241166  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:09.241334  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:09.241394  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:09.241845  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:14.243237  2728 slave.cpp:944] Authentication timed out
> W0719 12:29:14.244127  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:14.244290  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:14.244371  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:14.244820  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:19.245368  2726 slave.cpp:944] Authentication timed out
> W0719 12:29:19.245990  2727 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:19.246197  2727 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:19.246279  2727 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:19.246924  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:24.248062  2725 slave.cpp:944] Authentication timed out
> W0719 12:29:24.248940  2728 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:24.249105  2728 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:24.249163  2728 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:24.249614  2725 authenticatee.cpp:121] Creating new client
> SASL connection
> I0719 12:29:26.621341  2730 slave.cpp:4359] Current disk usage 1.87%. Max
> allowed age: 6.169248842791180days
> W0719 12:29:29.250006  2725 slave.cpp:944] Authentication timed out
> W0719 12:29:29.250661  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:29.250864  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:29.250923  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:29.251415  2731 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:34.252017  2729 slave.cpp:944] Authentication timed out
> W0719 12:29:34.252729  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:34.252897  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:34.252980  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:34.253417  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:39.254900  2725 slave.cpp:944] Authentication timed out
> W0719 12:29:39.255656  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:39.255822  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:39.255878  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:39.256358  2729 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:44.257105  2731 slave.cpp:944] Authentication timed out
> W0719 12:29:44.257777  2731 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:44.257964  2731 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:44.258023  2731 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:44.258486  2730 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:49.258904  2730 slave.cpp:944] Authentication timed out
> W0719 12:29:49.259932  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:49.260124  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:49.260187  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:49.260723  2726 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:54.261531  2730 slave.cpp:944] Authentication timed out
> W0719 12:29:54.262212  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:54.262387  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:54.262459  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:54.262948  2728 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:29:59.264312  2729 slave.cpp:944] Authentication timed out
> W0719 12:29:59.264811  2729 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:29:59.264956  2729 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:29:59.265014  2729 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:29:59.265439  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:30:04.266129  2729 slave.cpp:944] Authentication timed out
> W0719 12:30:04.266919  2725 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:30:04.267118  2725 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:30:04.267175  2725 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:30:04.267630  2727 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:30:09.268898  2730 slave.cpp:944] Authentication timed out
> W0719 12:30:09.269462  2730 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:30:09.269618  2730 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:30:09.269675  2730 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:30:09.270092  2732 authenticatee.cpp:121] Creating new client
> SASL connection
> W0719 12:30:14.271193  2726 slave.cpp:944] Authentication timed out
> W0719 12:30:14.271802  2726 slave.cpp:908] Failed to authenticate with
> master master@10.5.7.4:5050: Authentication discarded
> I0719 12:30:14.272011  2726 slave.cpp:859] Authenticating with master
> master@10.5.7.4:5050
> I0719 12:30:14.272073  2726 slave.cpp:864] Using default CRAM-MD5
> authenticatee
> I0719 12:30:14.272523  2732 authenticatee.cpp:121] Creating new client
> SASL connection
>
>
> On Mon, Jul 18, 2016 at 6:03 PM, Vinod Kone <vi...@apache.org> wrote:
>
>> Might be related to MESOS-2043
>> <https://issues.apache.org/jira/browse/MESOS-2043>?
>>
>> Can you paste master and agent logs?
>>
>> On Mon, Jul 18, 2016 at 3:13 PM, Douglas Nelson <it...@gmail.com>
>> wrote:
>>
>>> I have SSL enabled for mesos and for the most part everything seems to
>>> be working fine. But when I stop a slave node for long enough that it shows
>>> up with status LOST then I start up the slave again, registration with the
>>> master fails:
>>>
>>> I0718 15:51:45.646260 16791 master.cpp:5495] Authenticating slave(1)@
>>> 10.5.7.5:5051
>>> I0718 15:51:45.646960 16791 authenticator.cpp:98] Creating new server
>>> SASL connection
>>> I0718 15:51:50.648329 16790 master.cpp:5481] Queuing up authentication
>>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>>> progress
>>> W0718 15:51:50.648696 16790 master.cpp:5522] Failed to authenticate
>>> slave(1)@10.5.7.5:5051: Authentication discarded
>>>
>>> It cycles through this over and over again until I restart the master
>>> node. Is restarting the master the only way to handle re-authentication? I
>>> expected it to be more automatic. Thanks!
>>>
>>
>>
>

Re: Possible authentication bug

Posted by Douglas Nelson <it...@gmail.com>.

MASTER:

Log file created at: 2016/07/19 12:13:55
Running on machine: dev1
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0719 12:13:55.437517 11025 logging.cpp:188] INFO level logging started!
I0719 12:13:55.438292 11025 main.cpp:237] Build: 2016-07-08 16:57:43 by root
I0719 12:13:55.438343 11025 main.cpp:239] Version: 0.28.2
I0719 12:13:55.438967 11025 main.cpp:260] Using 'HierarchicalDRF' allocator
I0719 12:13:55.449280 11025 leveldb.cpp:174] Opened db in 9.870872ms
I0719 12:13:55.453482 11025 leveldb.cpp:181] Compacted db in 4.087025ms
I0719 12:13:55.453719 11025 leveldb.cpp:196] Created db iterator in 132186ns
I0719 12:13:55.453820 11025 leveldb.cpp:202] Seeked to beginning of db in
35181ns
I0719 12:13:55.454460 11025 leveldb.cpp:271] Iterated through 3 keys in the
db in 592652ns
I0719 12:13:55.454795 11025 replica.cpp:779] Replica recovered with log
positions 129 -> 130 with 0 holes and 0 unlearned
I0719 12:13:55.461314 11049 log.cpp:236] Attempting to join replica to
ZooKeeper group
I0719 12:13:55.465665 11025 main.cpp:471] Starting Mesos master
I0719 12:13:55.478101 11048 recover.cpp:447] Starting replica recovery
I0719 12:13:55.482789 11025 master.cpp:375] Master
c3c11abc-2f26-479f-8965-db34be5aa567 (dev1.something.com) started on
10.5.7.4:5050
I0719 12:13:55.482998 11025 master.cpp:377] Flags at startup:
--allocation_interval="1secs" --allocator="HierarchicalDRF"
--authenticate="true" --authenticate_http="true"
--authenticate_slaves="true" --authenticators="crammd5"
--authorizers="local" --credentials="/etc/mesos/auth/credentials"
--framework_sorter="drf" --help="false" --hostname_lookup="true"
--http_authenticators="basic" --initialize_driver_logging="true"
--log_auto_initialize="true" --log_dir="/var/log/mesos" --logbufsecs="0"
--logging_level="INFO" --max_completed_frameworks="50"
--max_completed_tasks_per_framework="1000" --max_slave_ping_timeouts="5"
--port="5050" --quiet="false" --quorum="1"
--recovery_slave_removal_limit="100%" --registry="replicated_log"
--registry_fetch_timeout="1mins" --registry_store_timeout="20secs"
--registry_strict="false" --root_submissions="true"
--slave_ping_timeout="15secs" --slave_reregister_timeout="10mins"
--user_sorter="drf" --version="false" --webui_dir="/usr/share/mesos/webui"
--work_dir="/var/lib/mesos" --zk="zk://10.5.7.4:2181/mesos"
--zk_session_timeout="10secs"
I0719 12:13:55.483695 11025 master.cpp:422] Master only allowing
authenticated frameworks to register
I0719 12:13:55.483753 11025 master.cpp:427] Master only allowing
authenticated slaves to register
I0719 12:13:55.483783 11025 credentials.hpp:35] Loading credentials for
authentication from '/etc/mesos/auth/credentials'
W0719 12:13:55.484066 11025 credentials.hpp:50] Permissions on credentials
file '/etc/mesos/auth/credentials' are too open. It is recommended that
your credentials file is NOT accessible by others.
I0719 12:13:55.484253 11048 recover.cpp:473] Replica is in VOTING status
I0719 12:13:55.484794 11048 recover.cpp:462] Recover process terminated
I0719 12:13:55.494889 11025 master.cpp:467] Using default 'crammd5'
authenticator
I0719 12:13:55.495386 11025 authenticator.cpp:518] Initializing server SASL
I0719 12:13:55.498042 11025 master.cpp:536] Using default 'basic' HTTP
authenticator
I0719 12:13:55.511752 11025 master.cpp:1650] Successfully attached file
'/var/log/mesos/mesos-master.INFO'
I0719 12:13:55.512787 11046 contender.cpp:147] Joining the ZK group
I0719 12:13:55.603083 11046 group.cpp:349] Group process (group(1)@
10.5.7.4:5050) connected to ZooKeeper
I0719 12:13:55.603395 11046 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (0, 0, 0)
I0719 12:13:55.603469 11046 group.cpp:427] Trying to create path
'/mesos/log_replicas' in ZooKeeper
I0719 12:13:55.610569 11051 group.cpp:349] Group process (group(4)@
10.5.7.4:5050) connected to ZooKeeper
I0719 12:13:55.610783 11051 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (0, 0, 0)
I0719 12:13:55.610844 11051 group.cpp:427] Trying to create path '/mesos'
in ZooKeeper
I0719 12:13:55.610780 11044 group.cpp:349] Group process (group(3)@
10.5.7.4:5050) connected to ZooKeeper
I0719 12:13:55.611119 11044 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (1, 0, 0)
I0719 12:13:55.611163 11044 group.cpp:427] Trying to create path '/mesos'
in ZooKeeper
I0719 12:13:55.611829 11049 group.cpp:349] Group process (group(2)@
10.5.7.4:5050) connected to ZooKeeper
I0719 12:13:55.611938 11049 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (1, 0, 0)
I0719 12:13:55.611970 11049 group.cpp:427] Trying to create path
'/mesos/log_replicas' in ZooKeeper
I0719 12:13:55.652545 11048 contender.cpp:263] New candidate (id='2') has
entered the contest for leadership
I0719 12:13:55.656810 11046 network.hpp:413] ZooKeeper group memberships
changed
I0719 12:13:55.657207 11044 group.cpp:700] Trying to get
'/mesos/log_replicas/0000000001' in ZooKeeper
I0719 12:13:55.659225 11049 detector.cpp:152] Detected a new leader:
(id='2')
I0719 12:13:55.660096 11051 group.cpp:700] Trying to get
'/mesos/json.info_0000000002' in ZooKeeper
I0719 12:13:55.663472 11050 network.hpp:461] ZooKeeper group PIDs: {
log-replica(1)@10.5.7.4:5050 }
I0719 12:13:55.664608 11046 detector.cpp:479] A new leading master (UPID=
master@10.5.7.4:5050) is detected
I0719 12:13:55.665132 11049 master.cpp:1711] The newly elected leader is
master@10.5.7.4:5050 with id c3c11abc-2f26-479f-8965-db34be5aa567
I0719 12:13:55.665240 11049 master.cpp:1724] Elected as the leading master!
I0719 12:13:55.665334 11049 master.cpp:1469] Recovering from registrar
I0719 12:13:55.665786 11044 registrar.cpp:307] Recovering registrar
I0719 12:13:55.668251 11049 log.cpp:659] Attempting to start the writer
I0719 12:13:55.674710 11045 replica.cpp:493] Replica received implicit
promise request from (6)@10.5.7.4:5050 with proposal 30
I0719 12:13:55.676144 11045 leveldb.cpp:304] Persisting metadata (8 bytes)
to leveldb took 1.29057ms
I0719 12:13:55.676283 11045 replica.cpp:342] Persisted promised to 30
I0719 12:13:55.678892 11044 coordinator.cpp:238] Coordinator attempting to
fill missing positions
I0719 12:13:55.679973 11047 log.cpp:675] Writer started with ending
position 130
I0719 12:13:55.684237 11051 leveldb.cpp:436] Reading position from leveldb
took 180882ns
I0719 12:13:55.684692 11051 leveldb.cpp:436] Reading position from leveldb
took 86170ns
I0719 12:13:55.689906 11045 registrar.cpp:340] Successfully fetched the
registry (320B) in 23.838976ms
I0719 12:13:55.690568 11045 registrar.cpp:439] Applied 1 operations in
253986ns; attempting to update the 'registry'
I0719 12:13:55.695715 11051 log.cpp:683] Attempting to append 359 bytes to
the log
I0719 12:13:55.696043 11045 coordinator.cpp:348] Coordinator attempting to
write APPEND action at position 131
I0719 12:13:55.698156 11045 replica.cpp:537] Replica received write request
for position 131 from (7)@10.5.7.4:5050
I0719 12:13:55.699312 11045 leveldb.cpp:341] Persisting action (379 bytes)
to leveldb took 918900ns
I0719 12:13:55.699405 11045 replica.cpp:712] Persisted action at 131
I0719 12:13:55.700897 11048 replica.cpp:691] Replica received learned
notice for position 131 from @0.0.0.0:0
I0719 12:13:55.701294 11048 leveldb.cpp:341] Persisting action (381 bytes)
to leveldb took 305226ns
I0719 12:13:55.701371 11048 replica.cpp:712] Persisted action at 131
I0719 12:13:55.701427 11048 replica.cpp:697] Replica learned APPEND action
at position 131
I0719 12:13:55.705004 11051 registrar.cpp:484] Successfully updated the
'registry' in 14.150144ms
I0719 12:13:55.705585 11051 registrar.cpp:370] Successfully recovered
registrar
I0719 12:13:55.705776 11047 log.cpp:702] Attempting to truncate the log to
131
I0719 12:13:55.706045 11046 coordinator.cpp:348] Coordinator attempting to
write TRUNCATE action at position 132
I0719 12:13:55.707181 11044 replica.cpp:537] Replica received write request
for position 132 from (8)@10.5.7.4:5050
I0719 12:13:55.707342 11046 master.cpp:1521] Recovered 1 slaves from the
Registry (320B) ; allowing 10mins for slaves to re-register
I0719 12:13:55.708180 11044 leveldb.cpp:341] Persisting action (18 bytes)
to leveldb took 889855ns
I0719 12:13:55.708283 11044 replica.cpp:712] Persisted action at 132
I0719 12:13:55.709445 11051 replica.cpp:691] Replica received learned
notice for position 132 from @0.0.0.0:0
I0719 12:13:55.709847 11051 leveldb.cpp:341] Persisting action (20 bytes)
to leveldb took 300975ns
I0719 12:13:55.709995 11051 leveldb.cpp:399] Deleting ~2 keys from leveldb
took 69236ns
I0719 12:13:55.710054 11051 replica.cpp:712] Persisted action at 132
I0719 12:13:55.710115 11051 replica.cpp:697] Replica learned TRUNCATE
action at position 132
I0719 12:14:05.059528 11047 master.cpp:5495] Authenticating
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:05.062243 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:14:05.098544 11049 authenticator.cpp:203] Received SASL
authentication start
I0719 12:14:05.098788 11049 authenticator.cpp:325] Authentication requires
more steps
I0719 12:14:05.107225 11044 authenticator.cpp:231] Received SASL
authentication step
I0719 12:14:05.107561 11044 authenticator.cpp:317] Authentication success
I0719 12:14:05.107964 11044 master.cpp:5525] Successfully authenticated
principal 'dev' at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:05.110188 11046 master.cpp:2231] Received SUBSCRIBE call for
framework 'marathon' at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:05.110599 11046 master.cpp:2302] Subscribing framework marathon
with checkpointing enabled and capabilities [  ]
I0719 12:14:05.112211 11051 hierarchical.cpp:265] Added framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:14:07.229689 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:14:07.250246 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:14:07.281565 11050 authenticator.cpp:203] Received SASL
authentication start
I0719 12:14:07.281793 11050 authenticator.cpp:325] Authentication requires
more steps
I0719 12:14:07.301153 11050 authenticator.cpp:231] Received SASL
authentication step
I0719 12:14:07.301486 11050 authenticator.cpp:317] Authentication success
I0719 12:14:07.302227 11051 master.cpp:5525] Successfully authenticated
principal 'dev' at slave(1)@10.5.7.5:5051
I0719 12:14:07.307806 11051 master.cpp:4439] Re-registering slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com)
I0719 12:14:07.310359 11044 registrar.cpp:439] Applied 1 operations in
176631ns; attempting to update the 'registry'
I0719 12:14:07.316319 11049 log.cpp:683] Attempting to append 359 bytes to
the log
I0719 12:14:07.316714 11046 coordinator.cpp:348] Coordinator attempting to
write APPEND action at position 133
I0719 12:14:07.318164 11046 replica.cpp:537] Replica received write request
for position 133 from (11)@10.5.7.4:5050
I0719 12:14:07.318792 11046 leveldb.cpp:341] Persisting action (379 bytes)
to leveldb took 519351ns
I0719 12:14:07.318903 11046 replica.cpp:712] Persisted action at 133
I0719 12:14:07.320116 11048 replica.cpp:691] Replica received learned
notice for position 133 from @0.0.0.0:0
I0719 12:14:07.320976 11048 leveldb.cpp:341] Persisting action (381 bytes)
to leveldb took 785461ns
I0719 12:14:07.321053 11048 replica.cpp:712] Persisted action at 133
I0719 12:14:07.321106 11048 replica.cpp:697] Replica learned APPEND action
at position 133
I0719 12:14:07.323272 11046 registrar.cpp:484] Successfully updated the
'registry' in 12.685056ms
I0719 12:14:07.323559 11048 log.cpp:702] Attempting to truncate the log to
133
I0719 12:14:07.323832 11049 coordinator.cpp:348] Coordinator attempting to
write TRUNCATE action at position 134
I0719 12:14:07.324877 11049 replica.cpp:537] Replica received write request
for position 134 from (12)@10.5.7.4:5050
I0719 12:14:07.325728 11049 leveldb.cpp:341] Persisting action (18 bytes)
to leveldb took 756749ns
I0719 12:14:07.325804 11049 replica.cpp:712] Persisted action at 134
W0719 12:14:07.326323 11047 master.cpp:6170] Possibly orphaned completed
task webapp.42f99194-4dda-11e6-8041-02421568c935 of framework
b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001 that ran on slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com)
I0719 12:14:07.326839 11049 replica.cpp:691] Replica received learned
notice for position 134 from @0.0.0.0:0
I0719 12:14:07.327742 11049 leveldb.cpp:341] Persisting action (20 bytes)
to leveldb took 717423ns
I0719 12:14:07.327713 11047 master.cpp:4521] Re-registered slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com) with cpus(*):4; mem(*):2929; disk(*):242457;
ports(*):[31000-32000]
I0719 12:14:07.328063 11047 master.cpp:4556] Sending updated checkpointed
resources  to slave ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@
10.5.7.5:5051 (dev2.something.com)
I0719 12:14:07.328157 11049 leveldb.cpp:399] Deleting ~2 keys from leveldb
took 188898ns
I0719 12:14:07.328233 11049 replica.cpp:712] Persisted action at 134
I0719 12:14:07.328300 11049 replica.cpp:697] Replica learned TRUNCATE
action at position 134
I0719 12:14:07.329198 11046 hierarchical.cpp:473] Added slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) with
cpus(*):4; mem(*):2929; disk(*):242457; ports(*):[31000-32000] (allocated: )
I0719 12:14:07.330371 11044 master.cpp:4618] Received update of slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com) with total oversubscribed resources
I0719 12:14:07.332751 11046 hierarchical.cpp:531] Slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) updated with
oversubscribed resources  (total: cpus(*):4; mem(*):2929; disk(*):242457;
ports(*):[31000-32000], allocated: cpus(*):4; mem(*):2929; disk(*):242457;
ports(*):[31000-32000])
I0719 12:14:07.332794 11047 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:07.432186 11045 master.cpp:3641] Processing DECLINE call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O0 ] for framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:09.503839 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:14:09.877415 11044 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:09.877856 11044 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:14:09.880434 11051 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:09.886831 11048 master.cpp:3641] Processing DECLINE call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O1 ] for framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:14.894800 11045 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:14.895149 11051 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:14:14.897150 11045 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:14.903162 11044 master.cpp:3641] Processing DECLINE call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O2 ] for framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:19.714318 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:14:19.914870 11049 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:19.915210 11047 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:14:19.917094 11049 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:19.923250 11051 master.cpp:3641] Processing DECLINE call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O3 ] for framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:20.019886 11050 master.cpp:4997] Performing implicit task state
reconciliation for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
(marathon) at scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:29.515525 11048 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:29.515820 11048 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:14:29.517138 11048 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:29.766815 11044 master.cpp:3104] Processing ACCEPT call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O4 ] on slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com) for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
(marathon) at scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:29.771697 11049 master.hpp:177] Adding task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
mem(*):32; ports(*):[31990-31990] on slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com)
I0719 12:14:29.772017 11049 master.cpp:3589] Launching task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 with
resources cpus(*):0.1; mem(*):32; ports(*):[31990-31990] on slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com)
I0719 12:14:30.589148 11045 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:30.595021 11046 master.cpp:3641] Processing DECLINE call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O5 ] for framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:14:30.610872 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:14:30.803971 11048 master.cpp:4763] Status update TASK_RUNNING
(UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 from slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com)
I0719 12:14:30.804188 11048 master.cpp:4811] Forwarding status update
TASK_RUNNING (UUID: 3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f) for task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:14:30.804988 11048 master.cpp:6421] Updating the state of task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_RUNNING,
status update state: TASK_RUNNING)
I0719 12:14:30.868975 11046 master.cpp:3918] Processing ACKNOWLEDGE call
3c3daeb0-fa48-41ad-b18d-6b415a6f5d4f for task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 on slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
I0719 12:14:40.864478 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:14:51.114728 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:01.365391 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:08.869536 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51290 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:18.887311 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:28.904626 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:38.918462 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:48.932732 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:15:58.946310 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:16:08.961182 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:16:18.976491 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:16:29.148555 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:16:30.951886 11046 master.cpp:5324] Sending 1 offers to framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:16:30.958127 11047 master.cpp:3641] Processing DECLINE call for
offers: [ c3c11abc-2f26-479f-8965-db34be5aa567-O6 ] for framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:16:39.398538 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:16:49.648846 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:16:59.899797 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:17:07.350857 11047 master.cpp:242] Shutting down slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 due to health check timeout
W0719 12:17:07.351191 11047 master.cpp:4930] Shutting down slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com) with message 'health check timed out'
I0719 12:17:07.351920 11047 master.cpp:6201] Removing slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com): health check timed out
I0719 12:17:07.352468 11047 master.cpp:6421] Updating the state of task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (latest state: TASK_LOST, status
update state: TASK_LOST)
I0719 12:17:07.352746 11045 hierarchical.cpp:505] Removed slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0
I0719 12:17:07.353837 11047 master.cpp:6487] Removing task
webapp.a27f8a59-4ddc-11e6-b368-02421568c935 with resources cpus(*):0.1;
mem(*):32; ports(*):[31990-31990] of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 on slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 at slave(1)@10.5.7.5:5051 (
dev2.something.com)
I0719 12:17:07.356307 11046 registrar.cpp:439] Applied 1 operations in
164323ns; attempting to update the 'registry'
I0719 12:17:07.360155 11045 log.cpp:683] Attempting to append 184 bytes to
the log
I0719 12:17:07.360399 11050 coordinator.cpp:348] Coordinator attempting to
write APPEND action at position 135
I0719 12:17:07.361938 11051 replica.cpp:537] Replica received write request
for position 135 from (40)@10.5.7.4:5050
I0719 12:17:07.362778 11051 leveldb.cpp:341] Persisting action (204 bytes)
to leveldb took 728456ns
I0719 12:17:07.362862 11051 replica.cpp:712] Persisted action at 135
I0719 12:17:07.363806 11051 replica.cpp:691] Replica received learned
notice for position 135 from @0.0.0.0:0
I0719 12:17:07.364334 11051 leveldb.cpp:341] Persisting action (206 bytes)
to leveldb took 430978ns
I0719 12:17:07.364414 11051 replica.cpp:712] Persisted action at 135
I0719 12:17:07.364471 11051 replica.cpp:697] Replica learned APPEND action
at position 135
I0719 12:17:07.366155 11047 registrar.cpp:484] Successfully updated the
'registry' in 9.689088ms
I0719 12:17:07.366540 11051 log.cpp:702] Attempting to truncate the log to
135
I0719 12:17:07.366657 11046 master.cpp:6320] Removed slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com): health check
timed out
I0719 12:17:07.366739 11047 coordinator.cpp:348] Coordinator attempting to
write TRUNCATE action at position 136
I0719 12:17:07.366789 11046 master.cpp:4806] Sending status update
TASK_LOST for task webapp.a27f8a59-4ddc-11e6-b368-02421568c935 of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 'Slave dev2.something.com
removed: health check timed out'
I0719 12:17:07.367236 11046 master.cpp:6343] Notifying framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462 of lost slave
ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0 (dev2.something.com) after
recovering
I0719 12:17:07.367993 11049 replica.cpp:537] Replica received write request
for position 136 from (41)@10.5.7.4:5050
I0719 12:17:07.368701 11049 leveldb.cpp:341] Persisting action (18 bytes)
to leveldb took 607673ns
I0719 12:17:07.368780 11049 replica.cpp:712] Persisted action at 136
I0719 12:17:07.369793 11048 replica.cpp:691] Replica received learned
notice for position 136 from @0.0.0.0:0
I0719 12:17:07.374290 11048 leveldb.cpp:341] Persisting action (20 bytes)
to leveldb took 4.333673ms
I0719 12:17:07.374531 11048 leveldb.cpp:399] Deleting ~2 keys from leveldb
took 142498ns
I0719 12:17:07.374651 11048 replica.cpp:712] Persisted action at 136
I0719 12:17:07.374763 11048 replica.cpp:697] Replica learned TRUNCATE
action at position 136
I0719 12:17:07.413800 11048 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:17:07.414114 11051 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:10.107131 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:17:12.435168 11051 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:17:12.435470 11051 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:17.455554 11049 master.cpp:3720] Processing REVIVE call for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:17:17.456195 11044 hierarchical.cpp:988] Removed offer filters for
framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:20.620965 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:17:28.907829 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:28.908550 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:17:30.634392 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:17:33.887810 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:17:33.888640 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:17:33.888864 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:33.889627 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:17:38.890091 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:17:38.890705 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:17:38.890908 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:38.891654 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:17:41.619175 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:17:43.892462 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:17:43.893380 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:17:43.893582 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:43.894261 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:17:48.894907 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:17:48.895597 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:17:48.895742 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:48.896458 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:17:52.622318 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:17:53.897758 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:17:53.898336 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:17:53.898510 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:53.899430 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:17:58.900116 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:17:58.900677 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:17:58.901741 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:17:58.902381 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:03.621628 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:18:03.902536 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:03.903519 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:03.903714 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:03.904594 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:08.904714 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:18:08.905208 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:08.905773 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:08.905913 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:08.906513 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:13.907012 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:18:13.907600 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:13.907958 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:13.908123 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:13.908871 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:14.624475 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:18:18.909356 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:18.910174 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:18.910326 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:18.911255 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:23.911778 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:18:23.912201 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:23.912739 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:23.913128 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:23.913684 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:25.624198 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:18:28.914320 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:28.914964 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:28.915148 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:28.915825 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:33.916498 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:33.917145 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:33.917282 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:33.918520 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:36.625072 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:18:38.918712 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:38.919631 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:38.920052 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:38.920876 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:43.921413 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:43.922125 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:43.922269 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:43.923209 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:47.627984 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:18:48.923180 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:48.923815 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:48.924010 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:48.924589 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:53.925963 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:53.926895 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:53.927124 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:53.927942 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:18:58.626113 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:18:58.927940 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:18:58.928613 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:18:58.928766 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:18:58.930127 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:03.930527 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:03.931396 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:03.931637 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:03.932543 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:08.932302 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:19:08.932852 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:08.933034 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:08.933212 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:08.933835 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:09.627965 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:19:13.934926 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:13.935514 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:13.935757 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:13.936638 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:18.936506 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:19:18.937036 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:18.937420 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:18.937613 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:18.938341 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:20.624949 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:19:23.938432 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:23.939040 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:23.939214 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:23.940078 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:28.941543 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:28.942198 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:28.942435 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:28.943140 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:31.627547 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:19:33.943943 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:19:33.944599 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:33.944835 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:33.944988 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:33.945693 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:38.945986 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:19:38.946478 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:38.946812 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:38.946949 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:38.947438 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:42.628363 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:19:43.948642 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:43.949349 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:43.949550 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:43.950319 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:48.951385 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:48.951884 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:48.952000 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:48.952672 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:53.628947 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:19:53.954244 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:53.954866 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:53.955090 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:53.956058 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:19:58.956197 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:19:58.956797 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:19:58.957006 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:19:58.957918 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:03.958680 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:03.959282 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:03.959450 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:03.960024 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:04.630003 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:20:08.961760 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:20:08.962337 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:08.962745 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:08.962887 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:08.963418 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:13.964138 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:13.964747 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:13.965031 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:13.965772 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:15.630905 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:20:18.966778 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:18.967433 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:18.967687 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:18.968503 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:23.969029 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:23.969832 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:23.969976 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:23.970935 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:26.628504 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:20:28.971153 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:20:28.971578 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:28.971874 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:28.972044 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:28.972594 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:33.973855 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:20:33.974304 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:33.974576 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:33.974689 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:33.975183 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:37.629881 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:20:38.976158 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:38.977018 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:38.977200 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:38.978510 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:43.977927 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:43.978523 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:43.978688 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:43.979673 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:48.631947 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:20:48.980355 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:48.981098 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:48.981258 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:48.982522 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:53.983012 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:53.983757 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:53.983986 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:53.984787 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:58.985410 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:20:58.985873 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:20:58.986018 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:20:58.986199 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:20:58.986820 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:20:59.630745 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:21:03.987254 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:03.987884 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:03.988030 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:03.989013 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:08.989428 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:21:08.990020 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:08.990285 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:08.990468 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:08.990981 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:10.633591 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:21:13.992058 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:13.992918 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:13.993191 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:13.994236 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:18.994022 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:21:18.994477 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:18.994962 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:18.995141 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:18.995689 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:21.647516 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:21:23.996325 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:23.997014 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:23.997248 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:23.998145 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:28.998471 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:28.999652 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:28.999872 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:29.000628 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:32.635963 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:21:34.000746 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:34.001461 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:34.001718 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:34.002413 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:39.003231 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:39.003926 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:39.004137 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:39.004909 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:43.639329 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:21:44.005383 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:44.005898 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:44.006666 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:44.007246 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:49.007254 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:49.007946 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:49.008180 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:49.009217 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:54.009101 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:54.009840 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:54.010089 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:54.010777 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:21:54.634322 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:21:59.011060 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:21:59.011662 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:21:59.011852 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:21:59.012444 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:04.013264 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:22:04.013864 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:04.014137 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:04.014339 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:04.015051 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:05.633569 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:22:09.015625 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:09.016332 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:09.016549 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:09.017400 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:14.018102 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:14.018810 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:14.018975 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:14.020356 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:16.638113 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:22:19.020917 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:22:19.021430 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:19.021756 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:19.021895 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:19.022616 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:24.023571 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:24.024431 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:24.024754 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:24.026060 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:27.643009 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:22:29.026162 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:29.026885 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:29.027034 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:29.028298 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:34.028923 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:34.029616 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:34.029865 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:34.030767 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:38.638548 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:22:39.031050 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:39.031689 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:39.031838 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:39.032711 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:44.033393 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:44.034443 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:44.034685 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:44.035321 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:49.035828 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:49.036345 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:49.036553 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:49.037240 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:49.636526 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:22:54.038525 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:22:54.039108 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:54.039549 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:54.039691 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:54.040293 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:22:59.040702 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:22:59.041216 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:22:59.041524 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:22:59.041749 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:22:59.042407 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:00.639822 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:23:04.042881 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:04.043432 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:04.043550 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:04.044637 11045 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:23:09.044903 11051 master.cpp:5541] Authentication timed out
W0719 12:23:09.045856 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:09.046164 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:09.046769 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:11.640506 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:23:14.047653 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:14.048249 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:14.048429 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:14.049170 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:19.049494 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:19.050199 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:19.050398 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:19.051105 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:22.639598 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:23:24.051370 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:24.052043 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:24.052255 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:24.053045 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:29.053254 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:23:29.053788 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:29.054085 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:29.054247 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:29.054867 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:33.639268 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:23:34.055438 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:23:34.055974 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:34.056213 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:34.056349 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:34.056926 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:39.057729 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:39.058291 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:39.058435 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:39.059099 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:44.059839 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:44.060503 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:44.060667 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:44.061758 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:44.639904 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:23:49.062311 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:49.062969 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:49.063107 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:49.063963 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:54.064702 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:54.065510 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:54.065732 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:54.066545 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:23:55.642850 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:23:59.067154 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:23:59.067770 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:23:59.068084 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:23:59.068683 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:04.069491 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:04.070219 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:04.070417 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:04.071239 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:06.643237 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:24:09.071045 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:09.071583 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:09.071717 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:09.072408 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:14.073580 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:14.074219 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:14.074439 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:14.075234 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:17.645912 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:24:19.076020 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:19.076668 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:19.077389 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:19.078135 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:20.012001 11051 master.cpp:5062] Performing explicit task state
reconciliation for 1 tasks of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 (marathon) at
scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:24:20.012976 11051 master.cpp:4997] Performing implicit task state
reconciliation for framework ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
(marathon) at scheduler-2c268b43-1bb7-44ba-8a7d-f915878c25e0@10.5.7.4:52462
I0719 12:24:24.078232 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:24.078850 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:24.079015 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:24.079833 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:28.644161 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:24:29.080739 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:29.081439 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:29.081719 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:29.082607 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:34.083586 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:24:34.084177 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:34.084424 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:34.085206 11048 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:24:39.085710 11044 master.cpp:5541] Authentication timed out
W0719 12:24:39.085863 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:39.086200 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:39.086688 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:39.645274 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:24:44.087625 11051 master.cpp:5541] Authentication timed out
W0719 12:24:44.087796 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:44.088065 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:44.088768 11051 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:24:49.089455 11046 master.cpp:5541] Authentication timed out
W0719 12:24:49.089862 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:49.090831 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:49.091661 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:24:50.645830 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:24:54.092033 11045 master.cpp:5541] Authentication timed out
W0719 12:24:54.092311 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:54.093353 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:54.093953 11045 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:24:59.094514 11045 master.cpp:5541] Authentication timed out
W0719 12:24:59.094877 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:24:59.095696 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:24:59.096318 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:01.646898 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:25:04.098826 11045 master.cpp:5541] Authentication timed out
I0719 12:25:04.099218 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:25:04.099647 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:04.099804 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:04.100505 11048 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:25:09.101160 11048 master.cpp:5541] Authentication timed out
W0719 12:25:09.101491 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:09.101846 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:09.102602 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:11.660732 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:25:14.102864 11046 master.cpp:5541] Authentication timed out
W0719 12:25:14.103103 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:14.103487 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:14.104300 11051 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:25:19.104631 11046 master.cpp:5541] Authentication timed out
W0719 12:25:19.104817 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:19.106343 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:19.106928 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:21.673857 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:25:24.107980 11050 master.cpp:5541] Authentication timed out
W0719 12:25:24.108252 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:24.109048 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:24.109756 11050 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:25:29.110285 11046 master.cpp:5541] Authentication timed out
W0719 12:25:29.110486 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:29.111874 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:29.112534 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:32.648926 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:25:34.113827 11046 master.cpp:5541] Authentication timed out
W0719 12:25:34.114099 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:34.114449 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:34.115272 11046 authenticator.cpp:98] Creating new server SASL
connection
W0719 12:25:39.115715 11048 master.cpp:5541] Authentication timed out
I0719 12:25:39.116320 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:25:39.116565 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:39.116698 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:39.117486 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:43.649667 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
W0719 12:25:44.117769 11050 master.cpp:5541] Authentication timed out
W0719 12:25:44.118093 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:44.119288 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:44.119913 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:49.121246 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:25:49.122274 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:49.122517 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:49.123277 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:54.123177 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:25:54.123744 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:54.123908 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:54.124613 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:25:54.650367 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:25:59.125444 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:25:59.126139 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:25:59.126317 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:25:59.127210 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:04.127732 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:04.128376 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:04.128587 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:04.129299 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:05.650473 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:26:09.129295 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:09.129942 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:09.130143 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:09.130893 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:14.132257 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:14.132984 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:14.133221 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:14.133903 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:16.650934 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:26:19.134830 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:26:19.135447 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:19.135695 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:19.135893 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:19.136826 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:24.137058 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:26:24.137684 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:24.137899 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:24.138236 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:24.138895 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:27.653244 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:26:29.139055 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:26:29.139578 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:29.139806 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:29.139937 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:29.140700 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:34.141217 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:34.142405 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:34.142554 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:34.143790 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:38.652467 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:26:39.143749 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:39.144413 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:39.144654 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:39.145241 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:44.146155 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:44.146859 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:44.147155 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:44.147868 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:49.148578 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:49.149238 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:49.149387 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:49.150285 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:49.652683 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:26:54.150988 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:26:54.151628 11047 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:54.152030 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:54.152228 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:54.152897 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:26:59.153410 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:26:59.153934 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:26:59.154147 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:26:59.154831 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:00.653033 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:27:04.155833 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:04.156525 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:04.156755 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:04.157620 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:09.158335 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:09.158910 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:09.159360 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:09.159893 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:11.654698 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:27:14.160549 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:14.161084 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:14.161367 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:14.162240 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:19.162997 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:19.163769 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:19.163976 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:19.165292 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:22.657127 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:27:24.165325 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:27:24.165812 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:24.166124 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:24.166335 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:24.166985 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:29.167835 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:29.168339 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:29.168457 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:29.169409 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:33.654701 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:27:34.170714 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:34.171576 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:34.171741 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:34.172905 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:39.173723 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:39.174795 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:39.174945 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:39.175669 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:44.175698 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:44.176259 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:44.176416 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:44.177232 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:44.655261 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:27:49.178457 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:27:49.178927 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:49.179082 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:49.179266 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:49.179967 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:54.181052 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:54.181738 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:54.181941 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:54.182667 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:27:55.656306 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:27:59.182857 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:27:59.183506 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:27:59.183732 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:27:59.184350 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:04.185437 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:04.186125 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:04.186264 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:04.187189 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:06.656854 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:28:09.188138 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:09.188856 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:09.189314 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:09.190248 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:14.190521 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:14.191016 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:14.191254 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:14.191875 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:17.659039 11047 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:28:19.192914 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:28:19.193475 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:19.193802 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:19.193939 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:19.194689 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:24.195497 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:24.196179 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:24.196390 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:24.197120 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:28.658488 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:28:29.198254 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:28:29.198724 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:29.198997 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:29.199115 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:29.199853 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:34.200729 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:28:34.201292 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:34.201460 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:34.201683 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:34.202527 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:39.202571 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:39.203177 11045 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:39.203344 11045 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:39.204205 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:39.659113 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:28:44.204735 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:44.205322 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:44.205466 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:44.206336 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:49.206876 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:28:49.207523 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:49.208042 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:49.208204 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:49.208876 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:50.659682 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:28:54.208958 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:54.209754 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:54.209985 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:54.210769 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:28:59.211021 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:28:59.211598 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:28:59.211750 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:28:59.212461 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:01.659540 11046 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:29:04.212937 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:04.213676 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:04.213832 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:04.214392 11051 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:09.215848 11050 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:09.216425 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:09.216625 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:09.217329 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:12.663728 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:29:14.218930 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:14.219707 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:14.220160 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:14.220715 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:19.221014 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:19.221758 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:19.221928 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:19.222981 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:23.660816 11048 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:29:24.223538 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:24.224192 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:24.224408 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:24.225181 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:29.225445 11045 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:29.226102 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:29.226274 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:29.227026 11050 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:34.227371 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:34.227906 11050 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:34.228116 11050 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:34.228906 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:34.662215 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:29:39.230406 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:39.231156 11051 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:39.231384 11051 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:39.232285 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:44.232631 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:44.233209 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:44.233371 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:44.234177 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:45.662411 11051 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:29:49.234805 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:49.235468 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:49.235636 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:49.237004 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:54.236991 11049 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:54.237576 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:54.237751 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:54.238709 11045 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:29:56.663393 11045 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:29:59.239316 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:29:59.239837 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:29:59.240001 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:29:59.240638 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:04.241629 11044 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:04.242250 11044 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:04.242472 11044 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:04.243203 11044 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:07.677323 11050 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:30:09.244133 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:09.244830 11046 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:09.245038 11046 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:09.245756 11046 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:14.246692 11046 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:14.247503 11049 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:14.247711 11049 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:14.248601 11049 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:18.664516 11044 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51253 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:30:19.248931 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:19.249675 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:19.249899 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:19.250809 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:24.250947 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:30:24.251588 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:24.251802 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:24.252038 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:24.252806 11048 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:29.253676 11051 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:29.254382 11047 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:29.254613 11047 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:29.255362 11047 authenticator.cpp:98] Creating new server SASL
connection
I0719 12:30:29.665109 11049 http.cpp:312] HTTP GET for /master/state from
10.5.1.91:51251 with User-Agent='Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36'
I0719 12:30:34.255038 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
I0719 12:30:34.255576 11048 master.cpp:5481] Queuing up authentication
request from slave(1)@10.5.7.5:5051 because authentication is still in
progress
W0719 12:30:34.255727 11048 master.cpp:5522] Failed to authenticate
slave(1)@10.5.7.5:5051: Authentication discarded
I0719 12:30:34.255898 11048 master.cpp:5495] Authenticating slave(1)@
10.5.7.5:5051
I0719 12:30:34.256418 11048 authenticator.cpp:98] Creating new server SASL
connection




AGENT:

Log file created at: 2016/07/19 12:17:26
Running on machine: dev2
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0719 12:17:26.442286  2706 logging.cpp:188] INFO level logging started!
I0719 12:17:26.442948  2706 main.cpp:223] Build: 2016-07-08 16:57:43 by root
I0719 12:17:26.443025  2706 main.cpp:225] Version: 0.28.2
I0719 12:17:26.563817  2706 containerizer.cpp:149] Using isolation:
posix/cpu,posix/mem,filesystem/posix
I0719 12:17:26.577617  2706 linux_launcher.cpp:101] Using
/sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
I0719 12:17:26.587507  2706 main.cpp:328] Starting Mesos slave
I0719 12:17:26.591217  2726 slave.cpp:193] Slave started on 1)@10.5.7.5:5051
I0719 12:17:26.591284  2726 slave.cpp:194] Flags at startup:
--appc_simple_discovery_uri_prefix="http://"
--appc_store_dir="/tmp/mesos/store/appc" --authenticatee="crammd5"
--cgroups_cpu_enable_pids_and_tids_count="false"
--cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
--cgroups_limit_swap="false" --cgroups_root="mesos"
--container_disk_watch_interval="15secs" --containerizers="docker,mesos"
--credential="/etc/mesos/auth/credential" --default_role="*"
--disk_watch_interval="1mins" --docker="docker"
--docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io"
--docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock"
--docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker"
--enforce_container_disk_quota="false"
--executor_environment_variables="{"SSL_CERT_FILE":"\/etc\/mesos\/ssl\/mesos.pem","SSL_ENABLED":"true","SSL_KEY_FILE":"\/etc\/mesos\/ssl\/mesos.key"}"
--executor_registration_timeout="5mins"
--executor_shutdown_grace_period="5secs"
--fetcher_cache_dir="/tmp/mesos/fetch" --fetcher_cache_size="2GB"
--frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
--hadoop_home="" --help="false" --hostname_lookup="true"
--image_provisioner_backend="copy" --initialize_driver_logging="true"
--isolation="posix/cpu,posix/mem" --launcher_dir="/usr/libexec/mesos"
--log_dir="/var/log/mesos" --logbufsecs="0" --logging_level="INFO"
--master="zk://10.5.7.4:2181/mesos"
--oversubscribed_resources_interval="15secs" --perf_duration="10secs"
--perf_interval="1mins" --port="5051" --qos_correction_interval_min="0ns"
--quiet="false" --recover="reconnect" --recovery_timeout="15mins"
--registration_backoff_factor="1secs" --revocable_cpu_low_priority="true"
--sandbox_directory="/mnt/mesos/sandbox" --strict="true"
--switch_user="true" --systemd_enable_support="true"
--systemd_runtime_directory="/run/systemd/system" --version="false"
--work_dir="/var/lib/mesos"
I0719 12:17:26.592231  2726 credentials.hpp:83] Loading credential for
authentication from '/etc/mesos/auth/credential'
W0719 12:17:26.592401  2726 credentials.hpp:98] Permissions on credential
file '/etc/mesos/auth/credential' are too open. It is recommended that your
credential file is NOT accessible by others.
I0719 12:17:26.593298  2730 group.cpp:349] Group process (group(1)@
10.5.7.5:5051) connected to ZooKeeper
I0719 12:17:26.593535  2730 group.cpp:831] Syncing group operations: queue
size (joins, cancels, datas) = (0, 0, 0)
I0719 12:17:26.593608  2730 group.cpp:427] Trying to create path '/mesos'
in ZooKeeper
I0719 12:17:26.600158  2725 detector.cpp:152] Detected a new leader:
(id='2')
I0719 12:17:26.600987  2727 group.cpp:700] Trying to get
'/mesos/json.info_0000000002' in ZooKeeper
I0719 12:17:26.603498  2726 slave.cpp:324] Slave using credential for: dev
I0719 12:17:26.604095  2731 detector.cpp:479] A new leading master (UPID=
master@10.5.7.4:5050) is detected
I0719 12:17:26.605425  2726 slave.cpp:464] Slave resources: cpus(*):4;
mem(*):2929; disk(*):242457; ports(*):[31000-32000]
I0719 12:17:26.605607  2726 slave.cpp:472] Slave attributes: [  ]
I0719 12:17:26.605659  2726 slave.cpp:477] Slave hostname:
dev2.something.com
I0719 12:17:26.612834  2729 state.cpp:58] Recovering state from
'/var/lib/mesos/meta'
I0719 12:17:26.630897  2731 slave.cpp:4638] Recovering framework
b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
I0719 12:17:26.631350  2731 slave.cpp:5461] Recovering executor
'webapp.42f99194-4dda-11e6-8041-02421568c935' of framework
b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
I0719 12:17:26.633368  2728 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935/runs/7b4fc655-5680-4581-8025-5c218de5e893'
for gc 6.99456443088days in the future
I0719 12:17:26.633872  2731 slave.cpp:4063] Cleaning up framework
b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
I0719 12:17:26.633875  2728 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935/runs/7b4fc655-5680-4581-8025-5c218de5e893'
for gc 6.99456442714074days in the future
I0719 12:17:26.634140  2728 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935'
for gc 6.99456442632days in the future
I0719 12:17:26.634322  2731 slave.cpp:4638] Recovering framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:26.634459  2728 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001/executors/webapp.42f99194-4dda-11e6-8041-02421568c935'
for gc 6.99456442578667days in the future
I0719 12:17:26.634485  2731 slave.cpp:5461] Recovering executor
'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:26.634722  2728 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001'
for gc 6.9999926607763days in the future
I0719 12:17:26.634914  2728 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/ed179d7c-d6b5-48ad-8644-abe43e21d4da-S0/frameworks/b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001'
for gc 6.99999265988444days in the future
I0719 12:17:26.634171  2729 status_update_manager.cpp:282] Closing status
update streams for framework b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
I0719 12:17:26.637401  2727 status_update_manager.cpp:200] Recovering
status update manager
I0719 12:17:26.637521  2727 status_update_manager.cpp:208] Recovering
executor 'webapp.42f99194-4dda-11e6-8041-02421568c935' of framework
b28941df-64ba-43a0-af47-b3ee6dbcdc91-0001
I0719 12:17:26.637789  2727 status_update_manager.cpp:208] Recovering
executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:26.642676  2726 docker.cpp:773] Recovering Docker containers
I0719 12:17:26.643991  2731 containerizer.cpp:407] Recovering containerizer
I0719 12:17:26.647322  2731 containerizer.cpp:456] Skipping recovery of
executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 because it was not launched from
mesos containerizer
I0719 12:17:26.654330  2727 provisioner.cpp:245] Provisioner recovery
complete
I0719 12:17:26.877194  2726 docker.cpp:877] Recovering container
'65fa6566-806b-407f-a273-ef3b94f6dede' for executor
'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
'ee03b9d8-797a-421b-9542-652e53fbc4fb-0000'
I0719 12:17:26.883574  2727 slave.cpp:4490] Sending reconnect request to
executor 'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000 at executor(1)@10.5.7.5:51307
I0719 12:17:26.900796  2728 slave.cpp:2799] Re-registering executor
'webapp.a27f8a59-4ddc-11e6-b368-02421568c935' of framework
ee03b9d8-797a-421b-9542-652e53fbc4fb-0000
I0719 12:17:26.993151  2729 docker.cpp:1427] Updated 'cpu.shares' to 204 at
/sys/fs/cgroup/cpu/docker/9efb4b79f889d69111e33dbce67bd62f8f82f3068d76a0b53118040f00f1ea5e
for container 65fa6566-806b-407f-a273-ef3b94f6dede
I0719 12:17:26.994645  2729 docker.cpp:1462] Updated
'memory.soft_limit_in_bytes' to 64MB for container
65fa6566-806b-407f-a273-ef3b94f6dede
I0719 12:17:26.997007  2729 docker.cpp:1488] Updated
'memory.limit_in_bytes' to 64MB at
/sys/fs/cgroup/memory/docker/9efb4b79f889d69111e33dbce67bd62f8f82f3068d76a0b53118040f00f1ea5e
for container 65fa6566-806b-407f-a273-ef3b94f6dede
I0719 12:17:28.886029  2725 slave.cpp:2952] Cleaning up un-reregistered
executors
I0719 12:17:28.886433  2725 slave.cpp:4550] Finished recovery
I0719 12:17:28.887312  2725 slave.cpp:4583] Garbage collecting old slave
0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0
I0719 12:17:28.887639  2725 slave.cpp:4583] Garbage collecting old slave
6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0
I0719 12:17:28.887648  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0' for gc
6.99998972913185days in the future
I0719 12:17:28.887792  2725 slave.cpp:4583] Garbage collecting old slave
31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1
I0719 12:17:28.887933  2725 slave.cpp:4583] Garbage collecting old slave
9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0
I0719 12:17:28.887967  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/0de52bae-57bd-46ea-b215-cccf1ecdeb4b-S0' for gc
6.99998972714074days in the future
I0719 12:17:28.888082  2725 slave.cpp:4583] Garbage collecting old slave
ca647578-5569-4990-bce2-2bf86ffacde3-S0
I0719 12:17:28.888191  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0' for gc
6.99998972571852days in the future
I0719 12:17:28.888355  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/6d14b8d9-ca9b-43d5-bf7c-2995cef2f3e4-S0' for gc
6.99998972497778days in the future
I0719 12:17:28.888499  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1' for gc
6.99998972407407days in the future
I0719 12:17:28.888689  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/31bdfaee-64f0-4f77-9dcb-20096ee123dc-S1' for gc
6.99998972341333days in the future
I0719 12:17:28.888829  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0' for gc
6.99998972240593days in the future
I0719 12:17:28.888967  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/9fc6a195-053e-459c-bb41-8cf3897e2cbf-S0' for gc
6.99998972164444days in the future
I0719 12:17:28.889101  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/slaves/ca647578-5569-4990-bce2-2bf86ffacde3-S0' for gc
6.99998972040593days in the future
I0719 12:17:28.889317  2732 gc.cpp:55] Scheduling
'/var/lib/mesos/meta/slaves/ca647578-5569-4990-bce2-2bf86ffacde3-S0' for gc
6.99998971916741days in the future
I0719 12:17:28.889744  2732 status_update_manager.cpp:174] Pausing sending
status updates
I0719 12:17:28.889770  2725 slave.cpp:796] New master detected at
master@10.5.7.4:5050
I0719 12:17:28.909875  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:28.909971  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:28.910511  2725 slave.cpp:832] Detecting new master
I0719 12:17:28.911711  2732 authenticatee.cpp:97] Initializing client SASL
I0719 12:17:28.914371  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:17:33.911911  2731 slave.cpp:944] Authentication timed out
W0719 12:17:33.912632  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:17:33.912788  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:33.912845  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:33.913310  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:17:38.914300  2730 slave.cpp:944] Authentication timed out
W0719 12:17:38.915067  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:17:38.915307  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:38.915366  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:38.915830  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:17:43.916915  2725 slave.cpp:944] Authentication timed out
W0719 12:17:43.917677  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:17:43.917862  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:43.917974  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:43.918507  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:17:48.918875  2730 slave.cpp:944] Authentication timed out
W0719 12:17:48.919649  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:17:48.919847  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:48.919919  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:48.920433  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:17:53.921869  2727 slave.cpp:944] Authentication timed out
W0719 12:17:53.922817  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:17:53.922976  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:53.923032  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:53.923563  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:17:58.924680  2732 slave.cpp:944] Authentication timed out
W0719 12:17:58.925246  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:17:58.925434  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:17:58.925498  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:17:58.926103  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:03.927134  2730 slave.cpp:944] Authentication timed out
W0719 12:18:03.927891  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:03.928052  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:03.928112  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:03.928536  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:08.929105  2728 slave.cpp:944] Authentication timed out
W0719 12:18:08.929832  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:08.930012  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:08.930093  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:08.930577  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:13.931543  2729 slave.cpp:944] Authentication timed out
W0719 12:18:13.932309  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:13.932477  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:13.932550  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:13.933042  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:18.933753  2731 slave.cpp:944] Authentication timed out
W0719 12:18:18.934530  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:18.934729  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:18.934792  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:18.935309  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:23.936511  2729 slave.cpp:944] Authentication timed out
W0719 12:18:23.937216  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:23.937366  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:23.937425  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:23.937896  2731 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:18:26.607225  2729 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169253812823067days
W0719 12:18:28.938818  2726 slave.cpp:944] Authentication timed out
W0719 12:18:28.939522  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:28.939715  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:28.939811  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:28.940253  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:33.941254  2725 slave.cpp:944] Authentication timed out
W0719 12:18:33.941874  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:33.942029  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:33.942085  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:33.942587  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:38.943189  2728 slave.cpp:944] Authentication timed out
W0719 12:18:38.943855  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:38.944048  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:38.944110  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:38.944572  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:43.945793  2725 slave.cpp:944] Authentication timed out
W0719 12:18:43.946396  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:43.946601  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:43.946729  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:43.947180  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:48.948164  2731 slave.cpp:944] Authentication timed out
W0719 12:18:48.948698  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:48.948873  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:48.948927  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:48.949345  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:53.950121  2727 slave.cpp:944] Authentication timed out
W0719 12:18:53.950748  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:53.951323  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:53.951418  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:53.951922  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:18:58.952540  2727 slave.cpp:944] Authentication timed out
W0719 12:18:58.953218  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:18:58.953387  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:18:58.953461  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:18:58.953919  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:03.954779  2729 slave.cpp:944] Authentication timed out
W0719 12:19:03.955401  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:03.955580  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:03.955657  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:03.956127  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:08.957002  2727 slave.cpp:944] Authentication timed out
W0719 12:19:08.957651  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:08.957834  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:08.957892  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:08.958351  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:13.959039  2731 slave.cpp:944] Authentication timed out
W0719 12:19:13.959734  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:13.959925  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:13.959991  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:13.960530  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:18.961040  2730 slave.cpp:944] Authentication timed out
W0719 12:19:18.961683  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:18.961946  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:18.962019  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:18.962450  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:23.963018  2725 slave.cpp:944] Authentication timed out
W0719 12:19:23.963678  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:23.963838  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:23.963894  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:23.964344  2729 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:19:26.608978  2729 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169253371042454days
W0719 12:19:28.965499  2726 slave.cpp:944] Authentication timed out
W0719 12:19:28.966660  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:28.966851  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:28.966958  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:28.967434  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:33.968541  2727 slave.cpp:944] Authentication timed out
W0719 12:19:33.969198  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:33.969369  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:33.969426  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:33.969883  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:38.970623  2732 slave.cpp:944] Authentication timed out
W0719 12:19:38.971215  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:38.971395  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:38.971454  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:38.971917  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:43.973112  2731 slave.cpp:944] Authentication timed out
W0719 12:19:43.973659  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:43.973954  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:43.974014  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:43.974584  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:48.975668  2731 slave.cpp:944] Authentication timed out
W0719 12:19:48.976529  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:48.976691  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:48.976758  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:48.977280  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:53.978732  2725 slave.cpp:944] Authentication timed out
W0719 12:19:53.979405  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:53.979600  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:53.979676  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:53.980168  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:19:58.980532  2726 slave.cpp:944] Authentication timed out
W0719 12:19:58.981240  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:19:58.981444  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:19:58.981531  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:19:58.982074  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:03.983041  2732 slave.cpp:944] Authentication timed out
W0719 12:20:03.983832  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:03.984200  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:03.984313  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:03.984802  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:08.986167  2726 slave.cpp:944] Authentication timed out
W0719 12:20:08.986836  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:08.986984  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:08.987040  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:08.987558  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:13.988708  2731 slave.cpp:944] Authentication timed out
W0719 12:20:13.989320  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:13.989475  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:13.989531  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:13.989976  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:18.991140  2729 slave.cpp:944] Authentication timed out
W0719 12:20:18.991741  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:18.991935  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:18.992003  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:18.992486  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:23.993616  2732 slave.cpp:944] Authentication timed out
W0719 12:20:23.994303  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:23.994451  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:23.994508  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:23.994971  2726 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:20:26.610416  2725 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169252929261840days
W0719 12:20:28.995944  2725 slave.cpp:944] Authentication timed out
W0719 12:20:28.996537  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:28.996763  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:28.996839  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:28.997261  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:33.998452  2726 slave.cpp:944] Authentication timed out
W0719 12:20:33.999094  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:33.999261  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:33.999356  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:33.999822  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:39.000432  2726 slave.cpp:944] Authentication timed out
W0719 12:20:39.001195  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:39.001364  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:39.001435  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:39.001938  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:44.002457  2732 slave.cpp:944] Authentication timed out
W0719 12:20:44.003163  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:44.003326  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:44.003382  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:44.003878  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:49.004884  2731 slave.cpp:944] Authentication timed out
W0719 12:20:49.005506  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:49.005669  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:49.005765  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:49.006237  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:54.007592  2726 slave.cpp:944] Authentication timed out
W0719 12:20:54.008363  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:54.008564  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:54.008625  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:54.009052  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:20:59.009913  2727 slave.cpp:944] Authentication timed out
W0719 12:20:59.010506  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:20:59.010984  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:20:59.011035  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:20:59.011549  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:04.012027  2725 slave.cpp:944] Authentication timed out
W0719 12:21:04.012603  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:04.012756  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:04.012812  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:04.013241  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:09.013691  2727 slave.cpp:944] Authentication timed out
W0719 12:21:09.014644  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:09.014842  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:09.014914  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:09.015367  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:14.016613  2727 slave.cpp:944] Authentication timed out
W0719 12:21:14.017313  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:14.017554  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:14.017611  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:14.018029  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:19.018718  2731 slave.cpp:944] Authentication timed out
W0719 12:21:19.019327  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:19.019505  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:19.019610  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:19.020059  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:24.020687  2729 slave.cpp:944] Authentication timed out
W0719 12:21:24.021451  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:24.021648  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:24.021708  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:24.022183  2732 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:21:26.611302  2729 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169252045700625days
W0719 12:21:29.023134  2732 slave.cpp:944] Authentication timed out
W0719 12:21:29.023824  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:29.023972  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:29.024029  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:29.024477  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:34.024953  2730 slave.cpp:944] Authentication timed out
W0719 12:21:34.025951  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:34.026146  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:34.026221  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:34.026741  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:39.027717  2731 slave.cpp:944] Authentication timed out
W0719 12:21:39.028527  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:39.028733  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:39.028796  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:39.029225  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:44.029718  2732 slave.cpp:944] Authentication timed out
W0719 12:21:44.030495  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:44.030712  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:44.030787  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:44.031285  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:49.031599  2726 slave.cpp:944] Authentication timed out
W0719 12:21:49.032201  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:49.032347  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:49.032402  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:49.033093  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:54.033625  2732 slave.cpp:944] Authentication timed out
W0719 12:21:54.034369  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:54.034518  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:54.034577  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:54.035073  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:21:59.035948  2729 slave.cpp:944] Authentication timed out
W0719 12:21:59.036551  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:21:59.036703  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:21:59.036761  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:21:59.037250  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:04.037871  2729 slave.cpp:944] Authentication timed out
W0719 12:22:04.038518  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:04.038727  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:04.038805  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:04.039253  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:09.040071  2726 slave.cpp:944] Authentication timed out
W0719 12:22:09.040676  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:09.040865  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:09.040998  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:09.041527  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:14.042394  2731 slave.cpp:944] Authentication timed out
W0719 12:22:14.043166  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:14.043345  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:14.043403  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:14.043896  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:19.045148  2731 slave.cpp:944] Authentication timed out
W0719 12:22:19.045684  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:19.046054  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:19.046166  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:19.046763  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:24.048012  2730 slave.cpp:944] Authentication timed out
W0719 12:22:24.048894  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:24.049060  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:24.049118  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:24.049540  2725 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:22:26.612486  2730 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169251935255463days
W0719 12:22:29.050376  2732 slave.cpp:944] Authentication timed out
W0719 12:22:29.051234  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:29.051421  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:29.051477  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:29.051931  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:34.053143  2730 slave.cpp:944] Authentication timed out
W0719 12:22:34.053758  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:34.054283  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:34.054349  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:34.054812  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:39.055835  2725 slave.cpp:944] Authentication timed out
W0719 12:22:39.056479  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:39.056671  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:39.056728  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:39.057149  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:44.057876  2727 slave.cpp:944] Authentication timed out
W0719 12:22:44.058470  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:44.058732  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:44.058861  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:44.059375  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:49.060206  2728 slave.cpp:944] Authentication timed out
W0719 12:22:49.061079  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:49.061347  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:49.061414  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:49.061935  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:54.062958  2729 slave.cpp:944] Authentication timed out
W0719 12:22:54.063654  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:54.063827  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:54.063910  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:54.064379  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:22:59.065388  2726 slave.cpp:944] Authentication timed out
W0719 12:22:59.065942  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:22:59.066133  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:22:59.066200  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:22:59.066651  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:04.067623  2726 slave.cpp:944] Authentication timed out
W0719 12:23:04.068518  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:04.068675  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:04.068732  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:04.069146  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:09.069924  2728 slave.cpp:944] Authentication timed out
W0719 12:23:09.070524  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:09.071009  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:09.071141  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:09.071633  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:14.072458  2726 slave.cpp:944] Authentication timed out
W0719 12:23:14.073009  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:14.073253  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:14.073353  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:14.073799  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:19.074239  2730 slave.cpp:944] Authentication timed out
W0719 12:23:19.074939  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:19.075140  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:19.075202  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:19.075644  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:24.076045  2731 slave.cpp:944] Authentication timed out
W0719 12:23:24.076644  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:24.076799  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:24.076855  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:24.077338  2729 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:23:26.613450  2731 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169251714365162days
W0719 12:23:29.077744  2732 slave.cpp:944] Authentication timed out
W0719 12:23:29.078488  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:29.078699  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:29.078765  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:29.079268  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:34.079994  2726 slave.cpp:944] Authentication timed out
W0719 12:23:34.080622  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:34.080775  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:34.080831  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:34.081279  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:39.082494  2732 slave.cpp:944] Authentication timed out
W0719 12:23:39.083204  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:39.083395  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:39.083456  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:39.083904  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:44.084430  2728 slave.cpp:944] Authentication timed out
W0719 12:23:44.085264  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:44.085417  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:44.085474  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:44.085904  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:49.086913  2732 slave.cpp:944] Authentication timed out
W0719 12:23:49.087604  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:49.087771  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:49.087828  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:49.088310  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:54.089237  2730 slave.cpp:944] Authentication timed out
W0719 12:23:54.089737  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:54.089936  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:54.090013  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:54.090585  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:23:59.091807  2729 slave.cpp:944] Authentication timed out
W0719 12:23:59.092536  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:23:59.092717  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:23:59.092798  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:23:59.093231  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:04.094005  2730 slave.cpp:944] Authentication timed out
W0719 12:24:04.094712  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:04.094916  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:04.094985  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:04.095445  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:09.095959  2731 slave.cpp:944] Authentication timed out
W0719 12:24:09.096580  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:09.096776  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:09.096835  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:09.097257  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:14.098148  2727 slave.cpp:944] Authentication timed out
W0719 12:24:14.098824  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:14.099046  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:14.099112  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:14.099601  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:19.100690  2725 slave.cpp:944] Authentication timed out
W0719 12:24:19.101272  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:19.101482  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:19.101557  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:19.102108  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:24.102840  2731 slave.cpp:944] Authentication timed out
W0719 12:24:24.103386  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:24.103564  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:24.103623  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:24.104046  2730 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:24:26.614562  2729 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169251272584549days
W0719 12:24:29.105397  2726 slave.cpp:944] Authentication timed out
W0719 12:24:29.105924  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:29.106081  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:29.106142  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:29.106680  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:34.108180  2727 slave.cpp:944] Authentication timed out
W0719 12:24:34.108861  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:34.109020  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:34.109076  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:34.109591  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:39.110515  2732 slave.cpp:944] Authentication timed out
W0719 12:24:39.111414  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:39.111649  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:39.111794  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:39.112321  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:44.112977  2726 slave.cpp:944] Authentication timed out
W0719 12:24:44.113622  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:44.113817  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:44.113931  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:44.114426  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:49.115818  2732 slave.cpp:944] Authentication timed out
W0719 12:24:49.116366  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:49.116511  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:49.116642  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:49.117146  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:54.117846  2731 slave.cpp:944] Authentication timed out
W0719 12:24:54.118671  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:54.118854  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:54.118928  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:54.119367  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:24:59.120332  2727 slave.cpp:944] Authentication timed out
W0719 12:24:59.120982  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:24:59.121171  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:24:59.121261  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:24:59.121898  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:04.123075  2728 slave.cpp:944] Authentication timed out
W0719 12:25:04.123872  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:04.124025  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:04.124083  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:04.124619  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:09.125983  2732 slave.cpp:944] Authentication timed out
W0719 12:25:09.126662  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:09.126938  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:09.127012  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:09.127579  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:14.128725  2725 slave.cpp:944] Authentication timed out
W0719 12:25:14.129214  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:14.129426  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:14.129490  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:14.129909  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:19.131160  2730 slave.cpp:944] Authentication timed out
W0719 12:25:19.131841  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:19.132031  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:19.132114  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:19.132634  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:24.133947  2729 slave.cpp:944] Authentication timed out
W0719 12:25:24.134681  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:24.134913  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:24.135006  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:24.135471  2727 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:25:26.616324  2731 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169250278578171days
W0719 12:25:29.136682  2726 slave.cpp:944] Authentication timed out
W0719 12:25:29.137389  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:29.137576  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:29.137644  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:29.138147  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:34.139066  2729 slave.cpp:944] Authentication timed out
W0719 12:25:34.139564  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:34.139700  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:34.139741  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:34.140192  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:39.141010  2727 slave.cpp:944] Authentication timed out
W0719 12:25:39.141593  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:39.141824  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:39.141904  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:39.142398  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:44.143599  2725 slave.cpp:944] Authentication timed out
W0719 12:25:44.144395  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:44.144596  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:44.144675  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:44.145153  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:49.145772  2730 slave.cpp:944] Authentication timed out
W0719 12:25:49.146502  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:49.146764  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:49.146833  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:49.147356  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:54.147753  2726 slave.cpp:944] Authentication timed out
W0719 12:25:54.148535  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:54.148747  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:54.148828  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:54.149282  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:25:59.150144  2726 slave.cpp:944] Authentication timed out
W0719 12:25:59.150773  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:25:59.150938  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:25:59.151005  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:25:59.151433  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:04.152122  2729 slave.cpp:944] Authentication timed out
W0719 12:26:04.152866  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:04.153053  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:04.153146  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:04.153659  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:09.154232  2730 slave.cpp:944] Authentication timed out
W0719 12:26:09.154769  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:09.154965  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:09.155025  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:09.155586  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:14.156872  2731 slave.cpp:944] Authentication timed out
W0719 12:26:14.157555  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:14.157750  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:14.157811  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:14.158351  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:19.159608  2725 slave.cpp:944] Authentication timed out
W0719 12:26:19.160323  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:19.160508  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:19.160570  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:19.160997  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:24.161856  2731 slave.cpp:944] Authentication timed out
W0719 12:26:24.162442  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:24.162650  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:24.162756  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:24.163245  2732 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:26:26.617436  2729 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169249836797558days
W0719 12:26:29.163816  2725 slave.cpp:944] Authentication timed out
W0719 12:26:29.164474  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:29.164662  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:29.164738  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:29.165218  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:34.165791  2728 slave.cpp:944] Authentication timed out
W0719 12:26:34.166488  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:34.166693  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:34.166795  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:34.167296  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:39.168243  2725 slave.cpp:944] Authentication timed out
W0719 12:26:39.168905  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:39.169137  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:39.169224  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:39.169710  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:44.170585  2729 slave.cpp:944] Authentication timed out
W0719 12:26:44.171337  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:44.171531  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:44.171602  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:44.172104  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:49.172936  2731 slave.cpp:944] Authentication timed out
W0719 12:26:49.173720  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:49.173892  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:49.173951  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:49.174450  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:54.175469  2729 slave.cpp:944] Authentication timed out
W0719 12:26:54.176153  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:54.176333  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:54.176393  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:54.176841  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:26:59.178102  2729 slave.cpp:944] Authentication timed out
W0719 12:26:59.178838  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:26:59.179049  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:26:59.179147  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:26:59.179589  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:04.180661  2726 slave.cpp:944] Authentication timed out
W0719 12:27:04.181210  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:04.181359  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:04.181416  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:04.181893  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:09.182955  2726 slave.cpp:944] Authentication timed out
W0719 12:27:09.183497  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:09.183751  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:09.183814  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:09.184252  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:14.184962  2730 slave.cpp:944] Authentication timed out
W0719 12:27:14.185637  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:14.185828  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:14.185912  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:14.186683  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:19.187254  2728 slave.cpp:944] Authentication timed out
W0719 12:27:19.188205  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:19.188354  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:19.188427  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:19.188961  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:24.189997  2728 slave.cpp:944] Authentication timed out
W0719 12:27:24.190727  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:24.190894  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:24.190964  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:24.191449  2727 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:27:26.618481  2729 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169249615907257days
W0719 12:27:29.192710  2727 slave.cpp:944] Authentication timed out
W0719 12:27:29.193230  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:29.193442  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:29.193518  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:29.193967  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:34.195250  2728 slave.cpp:944] Authentication timed out
W0719 12:27:34.195855  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:34.196043  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:34.196128  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:34.196566  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:39.197839  2726 slave.cpp:944] Authentication timed out
W0719 12:27:39.198698  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:39.198894  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:39.198969  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:39.199517  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:44.200739  2726 slave.cpp:944] Authentication timed out
W0719 12:27:44.201248  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:44.201396  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:44.201454  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:44.201923  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:49.203218  2725 slave.cpp:944] Authentication timed out
W0719 12:27:49.203873  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:49.204198  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:49.204267  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:49.204649  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:54.205327  2732 slave.cpp:944] Authentication timed out
W0719 12:27:54.206069  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:54.206326  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:54.206396  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:54.206984  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:27:59.207499  2728 slave.cpp:944] Authentication timed out
W0719 12:27:59.208132  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:27:59.208325  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:27:59.208370  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:27:59.208850  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:04.210011  2731 slave.cpp:944] Authentication timed out
W0719 12:28:04.210692  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:04.210889  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:04.210955  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:04.211443  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:09.212529  2732 slave.cpp:944] Authentication timed out
W0719 12:28:09.213176  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:09.213325  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:09.213460  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:09.214041  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:14.215133  2732 slave.cpp:944] Authentication timed out
W0719 12:28:14.215914  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:14.216078  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:14.216132  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:14.216625  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:19.217176  2726 slave.cpp:944] Authentication timed out
W0719 12:28:19.217996  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:19.218209  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:19.218291  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:19.218855  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:24.220111  2732 slave.cpp:944] Authentication timed out
W0719 12:28:24.220759  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:24.220952  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:24.221051  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:24.221539  2725 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:28:26.620170  2732 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169248732346030days
W0719 12:28:29.222877  2732 slave.cpp:944] Authentication timed out
W0719 12:28:29.223573  2732 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:29.223769  2732 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:29.223829  2732 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:29.224274  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:34.225024  2725 slave.cpp:944] Authentication timed out
W0719 12:28:34.225814  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:34.226063  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:34.226132  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:34.226645  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:39.227187  2726 slave.cpp:944] Authentication timed out
W0719 12:28:39.227819  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:39.227978  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:39.228101  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:39.228564  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:44.229616  2727 slave.cpp:944] Authentication timed out
W0719 12:28:44.230140  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:44.230283  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:44.230339  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:44.230813  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:49.231421  2731 slave.cpp:944] Authentication timed out
W0719 12:28:49.232055  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:49.232245  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:49.232305  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:49.232792  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:54.233731  2728 slave.cpp:944] Authentication timed out
W0719 12:28:54.234340  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:54.234493  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:54.234549  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:54.235028  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:28:59.235751  2728 slave.cpp:944] Authentication timed out
W0719 12:28:59.236279  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:28:59.236419  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:28:59.236475  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:28:59.236944  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:04.237779  2730 slave.cpp:944] Authentication timed out
W0719 12:29:04.238340  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:04.238487  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:04.238543  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:04.239001  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:09.240298  2728 slave.cpp:944] Authentication timed out
W0719 12:29:09.241166  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:09.241334  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:09.241394  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:09.241845  2725 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:14.243237  2728 slave.cpp:944] Authentication timed out
W0719 12:29:14.244127  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:14.244290  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:14.244371  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:14.244820  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:19.245368  2726 slave.cpp:944] Authentication timed out
W0719 12:29:19.245990  2727 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:19.246197  2727 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:19.246279  2727 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:19.246924  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:24.248062  2725 slave.cpp:944] Authentication timed out
W0719 12:29:24.248940  2728 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:24.249105  2728 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:24.249163  2728 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:24.249614  2725 authenticatee.cpp:121] Creating new client SASL
connection
I0719 12:29:26.621341  2730 slave.cpp:4359] Current disk usage 1.87%. Max
allowed age: 6.169248842791180days
W0719 12:29:29.250006  2725 slave.cpp:944] Authentication timed out
W0719 12:29:29.250661  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:29.250864  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:29.250923  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:29.251415  2731 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:34.252017  2729 slave.cpp:944] Authentication timed out
W0719 12:29:34.252729  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:34.252897  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:34.252980  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:34.253417  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:39.254900  2725 slave.cpp:944] Authentication timed out
W0719 12:29:39.255656  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:39.255822  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:39.255878  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:39.256358  2729 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:44.257105  2731 slave.cpp:944] Authentication timed out
W0719 12:29:44.257777  2731 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:44.257964  2731 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:44.258023  2731 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:44.258486  2730 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:49.258904  2730 slave.cpp:944] Authentication timed out
W0719 12:29:49.259932  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:49.260124  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:49.260187  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:49.260723  2726 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:54.261531  2730 slave.cpp:944] Authentication timed out
W0719 12:29:54.262212  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:54.262387  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:54.262459  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:54.262948  2728 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:29:59.264312  2729 slave.cpp:944] Authentication timed out
W0719 12:29:59.264811  2729 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:29:59.264956  2729 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:29:59.265014  2729 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:29:59.265439  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:30:04.266129  2729 slave.cpp:944] Authentication timed out
W0719 12:30:04.266919  2725 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:30:04.267118  2725 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:30:04.267175  2725 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:30:04.267630  2727 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:30:09.268898  2730 slave.cpp:944] Authentication timed out
W0719 12:30:09.269462  2730 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:30:09.269618  2730 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:30:09.269675  2730 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:30:09.270092  2732 authenticatee.cpp:121] Creating new client SASL
connection
W0719 12:30:14.271193  2726 slave.cpp:944] Authentication timed out
W0719 12:30:14.271802  2726 slave.cpp:908] Failed to authenticate with
master master@10.5.7.4:5050: Authentication discarded
I0719 12:30:14.272011  2726 slave.cpp:859] Authenticating with master
master@10.5.7.4:5050
I0719 12:30:14.272073  2726 slave.cpp:864] Using default CRAM-MD5
authenticatee
I0719 12:30:14.272523  2732 authenticatee.cpp:121] Creating new client SASL
connection


On Mon, Jul 18, 2016 at 6:03 PM, Vinod Kone <vi...@apache.org> wrote:

> Might be related to MESOS-2043
> <https://issues.apache.org/jira/browse/MESOS-2043>?
>
> Can you paste master and agent logs?
>
> On Mon, Jul 18, 2016 at 3:13 PM, Douglas Nelson <it...@gmail.com>
> wrote:
>
>> I have SSL enabled for mesos and for the most part everything seems to be
>> working fine. But when I stop a slave node for long enough that it shows up
>> with status LOST then I start up the slave again, registration with the
>> master fails:
>>
>> I0718 15:51:45.646260 16791 master.cpp:5495] Authenticating slave(1)@
>> 10.5.7.5:5051
>> I0718 15:51:45.646960 16791 authenticator.cpp:98] Creating new server
>> SASL connection
>> I0718 15:51:50.648329 16790 master.cpp:5481] Queuing up authentication
>> request from slave(1)@10.5.7.5:5051 because authentication is still in
>> progress
>> W0718 15:51:50.648696 16790 master.cpp:5522] Failed to authenticate
>> slave(1)@10.5.7.5:5051: Authentication discarded
>>
>> It cycles through this over and over again until I restart the master
>> node. Is restarting the master the only way to handle re-authentication? I
>> expected it to be more automatic. Thanks!
>>
>
>

Re: Possible authentication bug

Posted by Vinod Kone <vi...@apache.org>.

Might be related to MESOS-2043
<https://issues.apache.org/jira/browse/MESOS-2043>?

Can you paste master and agent logs?

On Mon, Jul 18, 2016 at 3:13 PM, Douglas Nelson <it...@gmail.com> wrote:

> I have SSL enabled for mesos and for the most part everything seems to be
> working fine. But when I stop a slave node for long enough that it shows up
> with status LOST then I start up the slave again, registration with the
> master fails:
>
> I0718 15:51:45.646260 16791 master.cpp:5495] Authenticating slave(1)@
> 10.5.7.5:5051
> I0718 15:51:45.646960 16791 authenticator.cpp:98] Creating new server SASL
> connection
> I0718 15:51:50.648329 16790 master.cpp:5481] Queuing up authentication
> request from slave(1)@10.5.7.5:5051 because authentication is still in
> progress
> W0718 15:51:50.648696 16790 master.cpp:5522] Failed to authenticate
> slave(1)@10.5.7.5:5051: Authentication discarded
>
> It cycles through this over and over again until I restart the master
> node. Is restarting the master the only way to handle re-authentication? I
> expected it to be more automatic. Thanks!
>