You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Neil Conway <ne...@gmail.com> on 2016/10/31 15:26:46 UTC
Re: Build failed in Jenkins: Mesos » autotools,gcc,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,centos:7,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2852
I spent a little while looking into this. The
"PersistentVolumeEndpointsTest.OfferCreateThenEndpointRemove" test
fails on the following expectations:
https://github.com/apache/mesos/blob/1e57459b7d3f571bdf18fec29b070e78ce719319/src/tests/persistent_volume_endpoints_tests.cpp#L1562
https://github.com/apache/mesos/blob/1e57459b7d3f571bdf18fec29b070e78ce719319/src/tests/persistent_volume_endpoints_tests.cpp#L1564
https://github.com/apache/mesos/blob/1e57459b7d3f571bdf18fec29b070e78ce719319/src/tests/persistent_volume_endpoints_tests.cpp#L1573
Which all seem quite innocent: similar or identical preamble code
occurs in many test cases. Looking at the log, it seems the scheduler
begins the authentication process but authentication times out:
12:27:56.527899 31618 sched.cpp:226] Version: 1.2.0
12:27:56.528548 31638 sched.cpp:330] New master detected at
master@172.17.0.2:48653
12:27:56.528661 31638 sched.cpp:396] Authenticating with master
master@172.17.0.2:48653
12:27:56.528681 31638 sched.cpp:403] Using default CRAM-MD5 authenticatee
12:28:01.529717 31637 sched.cpp:526] Authentication timed out
12:28:10.795253 31637 sched.cpp:466] Failed to authenticate with
master master@172.17.0.2:48653: Authentication discarded
In the scheduler driver, we fail the "authenticating" future at
12:28:01, but it is ~9 seconds before the associated `onAny` callback
is invoked to schedule retrying authentication; by the time the retry
backoff timeout expires, we've exceeded the 15 second Future timeout
in the test case.
Note that between 12:28:01.5 and 12:28:10.8, there is essentially
nothing happening:
W1031 12:28:01.529717 31637 sched.cpp:526] Authentication timed out
W1031 12:28:01.529752 31645 master.cpp:6789] Authentication timed out
I1031 12:28:10.794798 31652 status_update_manager.cpp:203] Recovering
status update manager
W1031 12:28:10.795033 31645 master.cpp:6769] Failed to authenticate
scheduler-877be3e9-9dc1-4de1-bf3e-a19b77b3d124@172.17.0.2:48653:
Authentication discarded
I1031 12:28:10.794939 31647 authenticator.cpp:432] Authentication
session cleanup for crammd5-authenticatee(655)@172.17.0.2:48653
I1031 12:28:10.795253 31637 sched.cpp:466] Failed to authenticate with
master master@172.17.0.2:48653: Authentication discarded
So I think the most likely culprit is VM lag.
We can try to workaround this by increasing some of the timeouts for
the test expectation futures, but of course that is just a kludge: if
we're going to experience random ~9.5 second VM-wide pauses, the tests
are likely to continue to be flaky unless we make more widespread
changes (e.g., increasing ALL expectation futures timeouts).
Neil
On Mon, Oct 31, 2016 at 8:34 AM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,COMPILER=gcc,CONFIGURATION=--verbose%20--enable-libevent%20--enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=centos%3A7,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-us1)&&(!ubuntu-6)/2852/changes>
>
> Changes:
>
> [alexr] Updated the stale comment in agent flags.
>
> ------------------------------------------
> [...truncated 219320 lines...]
> W1031 12:32:10.921492 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:10.921664 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:10.925060 31647 slave.cpp:208] Mesos agent started on (635)@172.17.0.2:48653
> I1031 12:32:10.925091 31647 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_xYMus0"
> I1031 12:32:10.925776 31647 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC/credential'
> I1031 12:32:10.925999 31647 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:10.926034 31647 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC/http_credentials'
> I1031 12:32:10.926313 31647 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:10.926491 31647 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:10.927832 31647 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:10.927932 31647 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:10.927968 31647 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:10.929754 31648 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_xYMus0/meta'
> I1031 12:32:10.930068 31642 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:10.930371 31646 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:10.931828 31640 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:10.932240 31648 slave.cpp:5399] Finished recovery
> I1031 12:32:10.932809 31648 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:10.933115 31641 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:10.934885 31643 process.cpp:3570] Handling HTTP event for process 'slave(635)' with path: '/slave(635)/monitor/statistics.json'
> I1031 12:32:10.935914 31648 http.cpp:839] Authorizing principal 'test-principal' to GET the '/monitor/statistics.json' endpoint
> I1031 12:32:10.939596 31640 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.AuthorizedRequest/1 (29 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.AuthorizedRequest/2
> I1031 12:32:10.949631 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:10.950292 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:10.950440 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:10.953157 31649 slave.cpp:208] Mesos agent started on (636)@172.17.0.2:48653
> I1031 12:32:10.953186 31649 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_YY07Ws"
> I1031 12:32:10.953714 31649 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64/credential'
> I1031 12:32:10.953852 31649 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:10.953874 31649 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64/http_credentials'
> I1031 12:32:10.954109 31649 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:10.954233 31649 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:10.955546 31649 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:10.955651 31649 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:10.955667 31649 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:10.957101 31652 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_YY07Ws/meta'
> I1031 12:32:10.957350 31647 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:10.957530 31652 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:10.958694 31647 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:10.959041 31646 slave.cpp:5399] Finished recovery
> I1031 12:32:10.959457 31646 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:10.959686 31649 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:10.961047 31637 process.cpp:3570] Handling HTTP event for process 'slave(636)' with path: '/slave(636)/containers'
> I1031 12:32:10.961745 31639 http.cpp:839] Authorizing principal 'test-principal' to GET the '/containers' endpoint
> I1031 12:32:10.964570 31641 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.AuthorizedRequest/2 (23 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/0
> I1031 12:32:10.974623 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:10.975366 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:10.975549 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:10.978215 31638 slave.cpp:208] Mesos agent started on (637)@172.17.0.2:48653
> I1031 12:32:10.978236 31638 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_oASJIV"
> I1031 12:32:10.978726 31638 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx/credential'
> I1031 12:32:10.978906 31638 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:10.978927 31638 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx/http_credentials'
> I1031 12:32:10.979166 31638 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:10.979367 31638 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:10.980345 31638 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:10.980422 31638 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:10.980434 31638 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:10.981706 31639 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_oASJIV/meta'
> I1031 12:32:10.981940 31642 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:10.982226 31649 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:10.983441 31650 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:10.983752 31637 slave.cpp:5399] Finished recovery
> I1031 12:32:10.984203 31637 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:10.984447 31640 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:10.985682 31640 process.cpp:3570] Handling HTTP event for process 'slave(637)' with path: '/slave(637)/monitor/statistics'
> I1031 12:32:10.986529 31652 http.cpp:839] Authorizing principal 'test-principal' to GET the '/monitor/statistics' endpoint
> I1031 12:32:10.989156 31651 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/0 (25 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/1
> I1031 12:32:10.999828 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:11.000572 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:11.000757 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:11.004053 31642 slave.cpp:208] Mesos agent started on (638)@172.17.0.2:48653
> I1031 12:32:11.004086 31642 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_iBewKo"
> I1031 12:32:11.004765 31642 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710/credential'
> I1031 12:32:11.004928 31642 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:11.004977 31642 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710/http_credentials'
> I1031 12:32:11.005246 31642 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:11.005403 31642 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:11.006629 31642 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:11.006709 31642 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:11.006723 31642 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:11.008316 31638 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_iBewKo/meta'
> I1031 12:32:11.008625 31647 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:11.008891 31649 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:11.010298 31637 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:11.010664 31646 slave.cpp:5399] Finished recovery
> I1031 12:32:11.011210 31646 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:11.011453 31639 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:11.012835 31637 process.cpp:3570] Handling HTTP event for process 'slave(638)' with path: '/slave(638)/monitor/statistics.json'
> I1031 12:32:11.014638 31647 http.cpp:839] Authorizing principal 'test-principal' to GET the '/monitor/statistics.json' endpoint
> I1031 12:32:11.017475 31639 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/1 (27 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/2
> I1031 12:32:11.026774 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:11.027331 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:11.027478 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:11.029881 31639 slave.cpp:208] Mesos agent started on (639)@172.17.0.2:48653
> I1031 12:32:11.029906 31639 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_8AWC3R"
> I1031 12:32:11.030359 31639 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu/credential'
> I1031 12:32:11.030477 31639 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:11.030496 31639 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu/http_credentials'
> I1031 12:32:11.030679 31639 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:11.030784 31639 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:11.031710 31639 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:11.031782 31639 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:11.031795 31639 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:11.032886 31644 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_8AWC3R/meta'
> I1031 12:32:11.033327 31637 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:11.033586 31639 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:11.035281 31637 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:11.035720 31641 slave.cpp:5399] Finished recovery
> I1031 12:32:11.036381 31641 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:11.036689 31639 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:11.038810 31642 process.cpp:3570] Handling HTTP event for process 'slave(639)' with path: '/slave(639)/containers'
> I1031 12:32:11.039907 31641 http.cpp:839] Authorizing principal 'test-principal' to GET the '/containers' endpoint
> I1031 12:32:11.043118 31641 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/2 (26 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.NoAuthorizer/0
> I1031 12:32:11.053463 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:11.054193 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:11.054376 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:11.055948 31618 cluster.cpp:435] Creating default 'local' authorizer
> I1031 12:32:11.057349 31641 slave.cpp:208] Mesos agent started on (640)@172.17.0.2:48653
> I1031 12:32:11.057425 31641 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_cCxODl"
> I1031 12:32:11.057895 31641 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/credential'
> I1031 12:32:11.058032 31641 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:11.058051 31641 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/http_credentials'
> I1031 12:32:11.058238 31641 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:11.058365 31641 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:11.059310 31641 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:11.059413 31641 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:11.059432 31641 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:11.061054 31639 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_cCxODl/meta'
> I1031 12:32:11.061524 31648 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:11.061805 31652 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:11.063594 31639 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:11.064234 31637 slave.cpp:5399] Finished recovery
> I1031 12:32:11.064710 31637 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:11.064916 31648 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:11.066171 31647 process.cpp:3570] Handling HTTP event for process 'slave(640)' with path: '/slave(640)/monitor/statistics'
> I1031 12:32:11.066920 31643 http.cpp:839] Authorizing principal 'test-principal' to GET the '/monitor/statistics' endpoint
> I1031 12:32:11.069689 31650 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/0 (26 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.NoAuthorizer/1
> I1031 12:32:11.080791 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:11.081549 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:11.081722 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:11.083326 31618 cluster.cpp:435] Creating default 'local' authorizer
> I1031 12:32:11.085261 31644 slave.cpp:208] Mesos agent started on (641)@172.17.0.2:48653
> I1031 12:32:11.085295 31644 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_01BHvP"
> I1031 12:32:11.086223 31644 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/credential'
> I1031 12:32:11.086376 31644 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:11.086401 31644 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/http_credentials'
> I1031 12:32:11.086669 31644 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:11.086786 31644 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:11.087756 31644 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:11.087832 31644 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:11.087846 31644 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:11.089253 31641 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_01BHvP/meta'
> I1031 12:32:11.089547 31646 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:11.089756 31638 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:11.090946 31647 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:11.091310 31640 slave.cpp:5399] Finished recovery
> I1031 12:32:11.091728 31640 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:11.091967 31651 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:11.093149 31647 process.cpp:3570] Handling HTTP event for process 'slave(641)' with path: '/slave(641)/monitor/statistics.json'
> I1031 12:32:11.093868 31651 http.cpp:839] Authorizing principal 'test-principal' to GET the '/monitor/statistics.json' endpoint
> I1031 12:32:11.096879 31643 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/1 (27 ms)
> [ RUN ] Endpoint/SlaveEndpointTest.NoAuthorizer/2
> I1031 12:32:11.108232 31618 containerizer.cpp:201] Using isolation: posix/cpu,posix/mem,filesystem/posix,network/cni
> W1031 12:32:11.109030 31618 backend.cpp:76] Failed to create 'aufs' backend: AufsBackend requires root privileges, but is running as user mesos
> W1031 12:32:11.109206 31618 backend.cpp:76] Failed to create 'bind' backend: BindBackend requires root privileges
> I1031 12:32:11.110757 31618 cluster.cpp:435] Creating default 'local' authorizer
> I1031 12:32:11.112639 31651 slave.cpp:208] Mesos agent started on (642)@172.17.0.2:48653
> I1031 12:32:11.112721 31651 slave.cpp:209] Flags at startup: --acls="" --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticatee="crammd5" --authentication_backoff_factor="1secs" --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false" --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup" --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs" --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/credential" --default_role="*" --disk_watch_interval="1mins" --docker="docker" --docker_kill_orphans="true" --docker_registry="https://registry-1.docker.io" --docker_remove_delay="6hrs" --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns" --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false" --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs" --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/fetch" --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1" --hadoop_home="" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_command_executor="false" --http_credentials="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/http_credentials" --image_provisioner_backend="copy" --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem" --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150" --oversubscribed_resources_interval="15secs" --perf_duration="10secs" --perf_interval="1mins" --qos_correction_interval_min="0ns" --quiet="false" --recover="reconnect" --recovery_timeout="15mins" --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true" --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW" --sandbox_directory="/mnt/mesos/sandbox" --strict="true" --switch_user="true" --systemd_enable_support="true" --systemd_runtime_directory="/run/systemd/system" --version="false" --work_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_ca6eFj"
> I1031 12:32:11.113402 31651 credentials.hpp:86] Loading credential for authentication from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/credential'
> I1031 12:32:11.113605 31651 slave.cpp:346] Agent using credential for: test-principal
> I1031 12:32:11.113634 31651 credentials.hpp:37] Loading credentials for authentication from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/http_credentials'
> I1031 12:32:11.113912 31651 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readonly'
> I1031 12:32:11.114121 31651 http.cpp:887] Using default 'basic' HTTP authenticator for realm 'mesos-agent-readwrite'
> I1031 12:32:11.115468 31651 slave.cpp:533] Agent resources: cpus(*):2; mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> I1031 12:32:11.115569 31651 slave.cpp:541] Agent attributes: [ ]
> I1031 12:32:11.115586 31651 slave.cpp:546] Agent hostname: 4f6c389cea92
> I1031 12:32:11.117015 31639 state.cpp:57] Recovering state from '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_ca6eFj/meta'
> I1031 12:32:11.117350 31645 status_update_manager.cpp:203] Recovering status update manager
> I1031 12:32:11.117545 31639 containerizer.cpp:557] Recovering containerizer
> I1031 12:32:11.119031 31642 provisioner.cpp:253] Provisioner recovery complete
> I1031 12:32:11.119446 31645 slave.cpp:5399] Finished recovery
> I1031 12:32:11.119987 31645 slave.cpp:5573] Querying resource estimator for oversubscribable resources
> I1031 12:32:11.120249 31637 slave.cpp:5587] Received oversubscribable resources {} from the resource estimator
> I1031 12:32:11.121784 31650 process.cpp:3570] Handling HTTP event for process 'slave(642)' with path: '/slave(642)/containers'
> I1031 12:32:11.122669 31638 http.cpp:839] Authorizing principal 'test-principal' to GET the '/containers' endpoint
> I1031 12:32:11.125516 31637 slave.cpp:787] Agent terminating
> [ OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (28 ms)
> [----------] 9 tests from Endpoint/SlaveEndpointTest (244 ms total)
>
> [----------] Global test environment tear-down
> [==========] 1364 tests from 154 test cases ran. (496175 ms total)
> [ PASSED ] 1363 tests.
> [ FAILED ] 1 test, listed below:
> [ FAILED ] PersistentVolumeEndpointsTest.OfferCreateThenEndpointRemove
>
> 1 FAILED TEST
> YOU HAVE 4 DISABLED TESTS
>
> make[4]: *** [check-local] Error 1
> make[4]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> make[3]: *** [check-am] Error 2
> make[3]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> make[2]: *** [check] Error 2
> make[2]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> make[1]: *** [check-recursive] Error 1
> make[1]: Leaving directory `/mesos/mesos-1.2.0/_build'
> make: *** [distcheck] Error 1
> + docker rmi mesos-1477914816-8359
> Untagged: mesos-1477914816-8359:latest
> Deleted: sha256:743f675d2801314957b97246616a10bcef411dc6752dfe322296fbaa50f72bb7
> Deleted: sha256:a61f526f8c9f48ecec48bd9050fc6a7a5e2ba332c35efc690df84a82abc063a3
> Deleted: sha256:0b6883066a366b0629fa447e80430736d62c3990b62d0a0624239ad320334f85
> Deleted: sha256:98962c1929ecc54bdb665dc3618489a9715e498372b2f7d6b4f2fce17b59f372
> Deleted: sha256:6a23bc318a3d60c5c232fb937743e5634018757a04e4c6f8314f52973d580d28
> Deleted: sha256:0b152ee148db28be05e4dcda3ded40b75114eabacefff4a5a0cbca7488c4f4c3
> Deleted: sha256:8fe85098485f161c5c63e44c56a150bcb57c2cde0c244181558d77ee84deae55
> Deleted: sha256:f4a348d46aba792d63b19fc4ffd131cdb2aa81b51db1b8aea72c0a03bfb3e8b9
> Deleted: sha256:e91aa4d35a2728de591f55e8a169de69213a3673eb1d57ae23fe9d792ca3377c
> Deleted: sha256:00fe97e4276dec4fe7ee1902517459aa589a82ffe3b44d546d40e298f5493922
> Deleted: sha256:7974bd2b5c27d1874ced6ed98612c65f6fc448adf0441619ab6a12b0202b7752
> Deleted: sha256:0dfb8541f30ca8280eec32e92b333a5259c964274470aef52c4d836df14f2300
> Deleted: sha256:38dedb84b0e91806c106ca60b55be0c250b5dd7a9835fff8cf947dffc1891fa7
> Deleted: sha256:fb08cf7282927f210c07fec3137a6a924ca9907e74642d541f0d7befa3d1e4fd
> Deleted: sha256:7e3d232c3781d955983847485fa90873007156c796b77e68acd5ce76c2c20d32
> Deleted: sha256:d1a76ebe400432a9c450bfcb736d6fb80f3f0438a66fabe5f1da8974de77978b
> Deleted: sha256:de5a6ce4bd8621028ad3d9c720bc4572a688d8e374445460acf8e6ab77b3ef78
> Deleted: sha256:d9280a65cd3a7ec1484a8205d8025feb23de44b081206a46822f5a212a7c6208
> Deleted: sha256:afd34f01a7c302fbcb6e94a11f4da60a0addfc86d30ef3943a1a0787220a4fb6
> Deleted: sha256:149bf2624cb44e56696305964dc3524af173347601181391442d15b073fa0ae5
> Deleted: sha256:cdab96d7c285fe42c3fe8d314e5c02196d2bffbecdd9c26a5b2faba849e79d7c
> Deleted: sha256:d0ef225f6403dbc4b68257bc21ae04f0b630aba21f05d0979a08f113b4a716d2
> Deleted: sha256:d58e420fd0a88f80c893435a8ffc9ed7be68d1c87a50d53c460a625d8b83735b
> Deleted: sha256:1c087b636db291826640ffb0d3ba26672b43494151ba4eef5955b2121a655a31
> Deleted: sha256:a07d988b44ae12da63d88902671fc5953854715358c10d7b4eb34bf0438b51f5
> Build step 'Execute shell' marked build as failure
Re: Build failed in Jenkins: Mesos » autotools,gcc,--verbose --enable-libevent --enable-ssl,GLOG_v=1 MESOS_VERBOSE=1,centos:7,(docker||Hadoop)&&(!ubuntu-us1)&&(!ubuntu-6) #2852
Posted by Alex Rukletsov <al...@mesosphere.com>.
Filed https://issues.apache.org/jira/browse/INFRA-12852
On Mon, Oct 31, 2016 at 4:26 PM, Neil Conway <ne...@gmail.com> wrote:
> I spent a little while looking into this. The
> "PersistentVolumeEndpointsTest.OfferCreateThenEndpointRemove" test
> fails on the following expectations:
>
> https://github.com/apache/mesos/blob/1e57459b7d3f571bdf18fec29b070e
> 78ce719319/src/tests/persistent_volume_endpoints_tests.cpp#L1562
> https://github.com/apache/mesos/blob/1e57459b7d3f571bdf18fec29b070e
> 78ce719319/src/tests/persistent_volume_endpoints_tests.cpp#L1564
> https://github.com/apache/mesos/blob/1e57459b7d3f571bdf18fec29b070e
> 78ce719319/src/tests/persistent_volume_endpoints_tests.cpp#L1573
>
> Which all seem quite innocent: similar or identical preamble code
> occurs in many test cases. Looking at the log, it seems the scheduler
> begins the authentication process but authentication times out:
>
> 12:27:56.527899 31618 sched.cpp:226] Version: 1.2.0
> 12:27:56.528548 31638 sched.cpp:330] New master detected at
> master@172.17.0.2:48653
> 12:27:56.528661 31638 sched.cpp:396] Authenticating with master
> master@172.17.0.2:48653
> 12:27:56.528681 31638 sched.cpp:403] Using default CRAM-MD5 authenticatee
> 12:28:01.529717 31637 sched.cpp:526] Authentication timed out
> 12:28:10.795253 31637 sched.cpp:466] Failed to authenticate with
> master master@172.17.0.2:48653: Authentication discarded
>
> In the scheduler driver, we fail the "authenticating" future at
> 12:28:01, but it is ~9 seconds before the associated `onAny` callback
> is invoked to schedule retrying authentication; by the time the retry
> backoff timeout expires, we've exceeded the 15 second Future timeout
> in the test case.
>
> Note that between 12:28:01.5 and 12:28:10.8, there is essentially
> nothing happening:
>
> W1031 12:28:01.529717 31637 sched.cpp:526] Authentication timed out
> W1031 12:28:01.529752 31645 master.cpp:6789] Authentication timed out
> I1031 12:28:10.794798 31652 status_update_manager.cpp:203] Recovering
> status update manager
> W1031 12:28:10.795033 31645 master.cpp:6769] Failed to authenticate
> scheduler-877be3e9-9dc1-4de1-bf3e-a19b77b3d124@172.17.0.2:48653:
> Authentication discarded
> I1031 12:28:10.794939 31647 authenticator.cpp:432] Authentication
> session cleanup for crammd5-authenticatee(655)@172.17.0.2:48653
> I1031 12:28:10.795253 31637 sched.cpp:466] Failed to authenticate with
> master master@172.17.0.2:48653: Authentication discarded
>
> So I think the most likely culprit is VM lag.
>
> We can try to workaround this by increasing some of the timeouts for
> the test expectation futures, but of course that is just a kludge: if
> we're going to experience random ~9.5 second VM-wide pauses, the tests
> are likely to continue to be flaky unless we make more widespread
> changes (e.g., increasing ALL expectation futures timeouts).
>
> Neil
>
>
> On Mon, Oct 31, 2016 at 8:34 AM, Apache Jenkins Server
> <je...@builds.apache.org> wrote:
> > See <https://builds.apache.org/job/Mesos/BUILDTOOL=autotools,
> COMPILER=gcc,CONFIGURATION=--verbose%20--enable-libevent%
> 20--enable-ssl,ENVIRONMENT=GLOG_v=1%20MESOS_VERBOSE=1,OS=
> centos%3A7,label_exp=(docker%7C%7CHadoop)&&(!ubuntu-us1)&&(
> !ubuntu-6)/2852/changes>
> >
> > Changes:
> >
> > [alexr] Updated the stale comment in agent flags.
> >
> > ------------------------------------------
> > [...truncated 219320 lines...]
> > W1031 12:32:10.921492 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:10.921664 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:10.925060 31647 slave.cpp:208] Mesos agent started on (635)@
> 172.17.0.2:48653
> > I1031 12:32:10.925091 31647 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_1_j6HfxC/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_1_j6HfxC/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --http_command_executor="false"
> --http_credentials="/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_1_j6HfxC/http_credentials" --image_provisioner_backend="copy"
> --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem"
> --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src"
> --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_j6HfxC"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_xYMus0"
> > I1031 12:32:10.925776 31647 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_1_j6HfxC/credential'
> > I1031 12:32:10.925999 31647 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:10.926034 31647 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_1_j6HfxC/http_credentials'
> > I1031 12:32:10.926313 31647 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:10.926491 31647 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:10.927832 31647 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:10.927932 31647 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:10.927968 31647 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:10.929754 31648 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_1_xYMus0/meta'
> > I1031 12:32:10.930068 31642 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:10.930371 31646 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:10.931828 31640 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:10.932240 31648 slave.cpp:5399] Finished recovery
> > I1031 12:32:10.932809 31648 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:10.933115 31641 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:10.934885 31643 process.cpp:3570] Handling HTTP event for
> process 'slave(635)' with path: '/slave(635)/monitor/statistics.json'
> > I1031 12:32:10.935914 31648 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/monitor/statistics.json' endpoint
> > I1031 12:32:10.939596 31640 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.AuthorizedRequest/1 (29 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.AuthorizedRequest/2
> > I1031 12:32:10.949631 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:10.950292 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:10.950440 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:10.953157 31649 slave.cpp:208] Mesos agent started on (636)@
> 172.17.0.2:48653
> > I1031 12:32:10.953186 31649 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_2_s8MF64/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_2_s8MF64/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --http_command_executor="false"
> --http_credentials="/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_2_s8MF64/http_credentials" --image_provisioner_backend="copy"
> --initialize_driver_logging="true" --isolation="posix/cpu,posix/mem"
> --launcher="posix" --launcher_dir="/mesos/mesos-1.2.0/_build/src"
> --logbufsecs="0" --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_s8MF64"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_YY07Ws"
> > I1031 12:32:10.953714 31649 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_2_s8MF64/credential'
> > I1031 12:32:10.953852 31649 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:10.953874 31649 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> AuthorizedRequest_2_s8MF64/http_credentials'
> > I1031 12:32:10.954109 31649 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:10.954233 31649 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:10.955546 31649 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:10.955651 31649 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:10.955667 31649 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:10.957101 31652 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_AuthorizedRequest_2_YY07Ws/meta'
> > I1031 12:32:10.957350 31647 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:10.957530 31652 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:10.958694 31647 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:10.959041 31646 slave.cpp:5399] Finished recovery
> > I1031 12:32:10.959457 31646 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:10.959686 31649 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:10.961047 31637 process.cpp:3570] Handling HTTP event for
> process 'slave(636)' with path: '/slave(636)/containers'
> > I1031 12:32:10.961745 31639 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/containers' endpoint
> > I1031 12:32:10.964570 31641 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.AuthorizedRequest/2 (23 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/0
> > I1031 12:32:10.974623 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:10.975366 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:10.975549 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:10.978215 31638 slave.cpp:208] Mesos agent started on (637)@
> 172.17.0.2:48653
> > I1031 12:32:10.978236 31638 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_0_8jeeWx/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_0_8jeeWx/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --http_command_executor="false"
> --http_credentials="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_0_8jeeWx/http_credentials"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0"
> --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_8jeeWx"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_oASJIV"
> > I1031 12:32:10.978726 31638 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_0_8jeeWx/credential'
> > I1031 12:32:10.978906 31638 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:10.978927 31638 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_0_8jeeWx/http_credentials'
> > I1031 12:32:10.979166 31638 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:10.979367 31638 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:10.980345 31638 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:10.980422 31638 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:10.980434 31638 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:10.981706 31639 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_0_oASJIV/meta'
> > I1031 12:32:10.981940 31642 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:10.982226 31649 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:10.983441 31650 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:10.983752 31637 slave.cpp:5399] Finished recovery
> > I1031 12:32:10.984203 31637 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:10.984447 31640 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:10.985682 31640 process.cpp:3570] Handling HTTP event for
> process 'slave(637)' with path: '/slave(637)/monitor/statistics'
> > I1031 12:32:10.986529 31652 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/monitor/statistics' endpoint
> > I1031 12:32:10.989156 31651 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/0 (25 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/1
> > I1031 12:32:10.999828 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:11.000572 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:11.000757 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:11.004053 31642 slave.cpp:208] Mesos agent started on (638)@
> 172.17.0.2:48653
> > I1031 12:32:11.004086 31642 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_1_2Nd710/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_1_2Nd710/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --http_command_executor="false"
> --http_credentials="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_1_2Nd710/http_credentials"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0"
> --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_2Nd710"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_iBewKo"
> > I1031 12:32:11.004765 31642 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_1_2Nd710/credential'
> > I1031 12:32:11.004928 31642 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:11.004977 31642 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_1_2Nd710/http_credentials'
> > I1031 12:32:11.005246 31642 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:11.005403 31642 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:11.006629 31642 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:11.006709 31642 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:11.006723 31642 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:11.008316 31638 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_1_iBewKo/meta'
> > I1031 12:32:11.008625 31647 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:11.008891 31649 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:11.010298 31637 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:11.010664 31646 slave.cpp:5399] Finished recovery
> > I1031 12:32:11.011210 31646 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:11.011453 31639 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:11.012835 31637 process.cpp:3570] Handling HTTP event for
> process 'slave(638)' with path: '/slave(638)/monitor/statistics.json'
> > I1031 12:32:11.014638 31647 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/monitor/statistics.json' endpoint
> > I1031 12:32:11.017475 31639 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/1 (27 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/2
> > I1031 12:32:11.026774 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:11.027331 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:11.027478 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:11.029881 31639 slave.cpp:208] Mesos agent started on (639)@
> 172.17.0.2:48653
> > I1031 12:32:11.029906 31639 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_2_wBZDpu/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_2_wBZDpu/fetch" --fetcher_cache_size="2GB"
> --frameworks_home="" --gc_delay="1weeks" --gc_disk_headroom="0.1"
> --hadoop_home="" --help="false" --hostname_lookup="true"
> --http_authenticators="basic" --http_command_executor="false"
> --http_credentials="/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_2_wBZDpu/http_credentials"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0"
> --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_wBZDpu"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_8AWC3R"
> > I1031 12:32:11.030359 31639 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_2_wBZDpu/credential'
> > I1031 12:32:11.030477 31639 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:11.030496 31639 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> UnauthorizedRequest_2_wBZDpu/http_credentials'
> > I1031 12:32:11.030679 31639 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:11.030784 31639 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:11.031710 31639 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:11.031782 31639 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:11.031795 31639 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:11.032886 31644 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_UnauthorizedRequest_2_8AWC3R/meta'
> > I1031 12:32:11.033327 31637 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:11.033586 31639 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:11.035281 31637 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:11.035720 31641 slave.cpp:5399] Finished recovery
> > I1031 12:32:11.036381 31641 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:11.036689 31639 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:11.038810 31642 process.cpp:3570] Handling HTTP event for
> process 'slave(639)' with path: '/slave(639)/containers'
> > I1031 12:32:11.039907 31641 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/containers' endpoint
> > I1031 12:32:11.043118 31641 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.UnauthorizedRequest/2 (26 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.NoAuthorizer/0
> > I1031 12:32:11.053463 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:11.054193 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:11.054376 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:11.055948 31618 cluster.cpp:435] Creating default 'local'
> authorizer
> > I1031 12:32:11.057349 31641 slave.cpp:208] Mesos agent started on (640)@
> 172.17.0.2:48653
> > I1031 12:32:11.057425 31641 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_0_xyy63X/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/fetch"
> --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
> --gc_disk_headroom="0.1" --hadoop_home="" --help="false"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_command_executor="false" --http_credentials="/tmp/
> Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X/http_credentials"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0"
> --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_xyy63X"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_cCxODl"
> > I1031 12:32:11.057895 31641 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_0_xyy63X/credential'
> > I1031 12:32:11.058032 31641 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:11.058051 31641 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_0_xyy63X/http_credentials'
> > I1031 12:32:11.058238 31641 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:11.058365 31641 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:11.059310 31641 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:11.059413 31641 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:11.059432 31641 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:11.061054 31639 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_0_cCxODl/meta'
> > I1031 12:32:11.061524 31648 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:11.061805 31652 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:11.063594 31639 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:11.064234 31637 slave.cpp:5399] Finished recovery
> > I1031 12:32:11.064710 31637 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:11.064916 31648 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:11.066171 31647 process.cpp:3570] Handling HTTP event for
> process 'slave(640)' with path: '/slave(640)/monitor/statistics'
> > I1031 12:32:11.066920 31643 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/monitor/statistics' endpoint
> > I1031 12:32:11.069689 31650 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/0 (26 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.NoAuthorizer/1
> > I1031 12:32:11.080791 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:11.081549 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:11.081722 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:11.083326 31618 cluster.cpp:435] Creating default 'local'
> authorizer
> > I1031 12:32:11.085261 31644 slave.cpp:208] Mesos agent started on (641)@
> 172.17.0.2:48653
> > I1031 12:32:11.085295 31644 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_1_a3gp0r/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/fetch"
> --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
> --gc_disk_headroom="0.1" --hadoop_home="" --help="false"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_command_executor="false" --http_credentials="/tmp/
> Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r/http_credentials"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0"
> --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_a3gp0r"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_01BHvP"
> > I1031 12:32:11.086223 31644 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_1_a3gp0r/credential'
> > I1031 12:32:11.086376 31644 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:11.086401 31644 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_1_a3gp0r/http_credentials'
> > I1031 12:32:11.086669 31644 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:11.086786 31644 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:11.087756 31644 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:11.087832 31644 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:11.087846 31644 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:11.089253 31641 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_1_01BHvP/meta'
> > I1031 12:32:11.089547 31646 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:11.089756 31638 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:11.090946 31647 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:11.091310 31640 slave.cpp:5399] Finished recovery
> > I1031 12:32:11.091728 31640 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:11.091967 31651 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:11.093149 31647 process.cpp:3570] Handling HTTP event for
> process 'slave(641)' with path: '/slave(641)/monitor/statistics.json'
> > I1031 12:32:11.093868 31651 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/monitor/statistics.json' endpoint
> > I1031 12:32:11.096879 31643 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/1 (27 ms)
> > [ RUN ] Endpoint/SlaveEndpointTest.NoAuthorizer/2
> > I1031 12:32:11.108232 31618 containerizer.cpp:201] Using isolation:
> posix/cpu,posix/mem,filesystem/posix,network/cni
> > W1031 12:32:11.109030 31618 backend.cpp:76] Failed to create 'aufs'
> backend: AufsBackend requires root privileges, but is running as user mesos
> > W1031 12:32:11.109206 31618 backend.cpp:76] Failed to create 'bind'
> backend: BindBackend requires root privileges
> > I1031 12:32:11.110757 31618 cluster.cpp:435] Creating default 'local'
> authorizer
> > I1031 12:32:11.112639 31651 slave.cpp:208] Mesos agent started on (642)@
> 172.17.0.2:48653
> > I1031 12:32:11.112721 31651 slave.cpp:209] Flags at startup: --acls=""
> --appc_simple_discovery_uri_prefix="http://" --appc_store_dir="/tmp/mesos/store/appc"
> --authenticate_http_readonly="true" --authenticate_http_readwrite="true"
> --authenticatee="crammd5" --authentication_backoff_factor="1secs"
> --authorizer="local" --cgroups_cpu_enable_pids_and_tids_count="false"
> --cgroups_enable_cfs="false" --cgroups_hierarchy="/sys/fs/cgroup"
> --cgroups_limit_swap="false" --cgroups_root="mesos" --container_disk_watch_interval="15secs"
> --containerizers="mesos" --credential="/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_2_4L2meW/credential" --default_role="*"
> --disk_watch_interval="1mins" --docker="docker"
> --docker_kill_orphans="true" --docker_registry="https://
> registry-1.docker.io" --docker_remove_delay="6hrs"
> --docker_socket="/var/run/docker.sock" --docker_stop_timeout="0ns"
> --docker_store_dir="/tmp/mesos/store/docker" --docker_volume_checkpoint_
> dir="/var/run/mesos/isolators/docker/volume" --enforce_container_disk_quota="false"
> --executor_registration_timeout="1mins" --executor_shutdown_grace_period="5secs"
> --fetcher_cache_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/fetch"
> --fetcher_cache_size="2GB" --frameworks_home="" --gc_delay="1weeks"
> --gc_disk_headroom="0.1" --hadoop_home="" --help="false"
> --hostname_lookup="true" --http_authenticators="basic"
> --http_command_executor="false" --http_credentials="/tmp/
> Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW/http_credentials"
> --image_provisioner_backend="copy" --initialize_driver_logging="true"
> --isolation="posix/cpu,posix/mem" --launcher="posix"
> --launcher_dir="/mesos/mesos-1.2.0/_build/src" --logbufsecs="0"
> --logging_level="INFO" --max_completed_executors_per_framework="150"
> --oversubscribed_resources_interval="15secs" --perf_duration="10secs"
> --perf_interval="1mins" --qos_correction_interval_min="0ns"
> --quiet="false" --recover="reconnect" --recovery_timeout="15mins"
> --registration_backoff_factor="10ms" --resources="cpus:2;gpus:0;
> mem:1024;disk:1024;ports:[31000-32000]" --revocable_cpu_low_priority="true"
> --runtime_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_4L2meW"
> --sandbox_directory="/mnt/mesos/sandbox" --strict="true"
> --switch_user="true" --systemd_enable_support="true"
> --systemd_runtime_directory="/run/systemd/system" --version="false"
> --work_dir="/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_ca6eFj"
> > I1031 12:32:11.113402 31651 credentials.hpp:86] Loading credential for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_2_4L2meW/credential'
> > I1031 12:32:11.113605 31651 slave.cpp:346] Agent using credential for:
> test-principal
> > I1031 12:32:11.113634 31651 credentials.hpp:37] Loading credentials for
> authentication from '/tmp/Endpoint_SlaveEndpointTest_
> NoAuthorizer_2_4L2meW/http_credentials'
> > I1031 12:32:11.113912 31651 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readonly'
> > I1031 12:32:11.114121 31651 http.cpp:887] Using default 'basic' HTTP
> authenticator for realm 'mesos-agent-readwrite'
> > I1031 12:32:11.115468 31651 slave.cpp:533] Agent resources: cpus(*):2;
> mem(*):1024; disk(*):1024; ports(*):[31000-32000]
> > I1031 12:32:11.115569 31651 slave.cpp:541] Agent attributes: [ ]
> > I1031 12:32:11.115586 31651 slave.cpp:546] Agent hostname: 4f6c389cea92
> > I1031 12:32:11.117015 31639 state.cpp:57] Recovering state from
> '/tmp/Endpoint_SlaveEndpointTest_NoAuthorizer_2_ca6eFj/meta'
> > I1031 12:32:11.117350 31645 status_update_manager.cpp:203] Recovering
> status update manager
> > I1031 12:32:11.117545 31639 containerizer.cpp:557] Recovering
> containerizer
> > I1031 12:32:11.119031 31642 provisioner.cpp:253] Provisioner recovery
> complete
> > I1031 12:32:11.119446 31645 slave.cpp:5399] Finished recovery
> > I1031 12:32:11.119987 31645 slave.cpp:5573] Querying resource estimator
> for oversubscribable resources
> > I1031 12:32:11.120249 31637 slave.cpp:5587] Received oversubscribable
> resources {} from the resource estimator
> > I1031 12:32:11.121784 31650 process.cpp:3570] Handling HTTP event for
> process 'slave(642)' with path: '/slave(642)/containers'
> > I1031 12:32:11.122669 31638 http.cpp:839] Authorizing principal
> 'test-principal' to GET the '/containers' endpoint
> > I1031 12:32:11.125516 31637 slave.cpp:787] Agent terminating
> > [ OK ] Endpoint/SlaveEndpointTest.NoAuthorizer/2 (28 ms)
> > [----------] 9 tests from Endpoint/SlaveEndpointTest (244 ms total)
> >
> > [----------] Global test environment tear-down
> > [==========] 1364 tests from 154 test cases ran. (496175 ms total)
> > [ PASSED ] 1363 tests.
> > [ FAILED ] 1 test, listed below:
> > [ FAILED ] PersistentVolumeEndpointsTest.OfferCreateThenEndpointRemove
> >
> > 1 FAILED TEST
> > YOU HAVE 4 DISABLED TESTS
> >
> > make[4]: *** [check-local] Error 1
> > make[4]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> > make[3]: *** [check-am] Error 2
> > make[3]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> > make[2]: *** [check] Error 2
> > make[2]: Leaving directory `/mesos/mesos-1.2.0/_build/src'
> > make[1]: *** [check-recursive] Error 1
> > make[1]: Leaving directory `/mesos/mesos-1.2.0/_build'
> > make: *** [distcheck] Error 1
> > + docker rmi mesos-1477914816-8359
> > Untagged: mesos-1477914816-8359:latest
> > Deleted: sha256:743f675d2801314957b97246616a10
> bcef411dc6752dfe322296fbaa50f72bb7
> > Deleted: sha256:a61f526f8c9f48ecec48bd9050fc6a
> 7a5e2ba332c35efc690df84a82abc063a3
> > Deleted: sha256:0b6883066a366b0629fa447e804307
> 36d62c3990b62d0a0624239ad320334f85
> > Deleted: sha256:98962c1929ecc54bdb665dc3618489
> a9715e498372b2f7d6b4f2fce17b59f372
> > Deleted: sha256:6a23bc318a3d60c5c232fb937743e5
> 634018757a04e4c6f8314f52973d580d28
> > Deleted: sha256:0b152ee148db28be05e4dcda3ded40
> b75114eabacefff4a5a0cbca7488c4f4c3
> > Deleted: sha256:8fe85098485f161c5c63e44c56a150
> bcb57c2cde0c244181558d77ee84deae55
> > Deleted: sha256:f4a348d46aba792d63b19fc4ffd131
> cdb2aa81b51db1b8aea72c0a03bfb3e8b9
> > Deleted: sha256:e91aa4d35a2728de591f55e8a169de
> 69213a3673eb1d57ae23fe9d792ca3377c
> > Deleted: sha256:00fe97e4276dec4fe7ee1902517459
> aa589a82ffe3b44d546d40e298f5493922
> > Deleted: sha256:7974bd2b5c27d1874ced6ed98612c6
> 5f6fc448adf0441619ab6a12b0202b7752
> > Deleted: sha256:0dfb8541f30ca8280eec32e92b333a
> 5259c964274470aef52c4d836df14f2300
> > Deleted: sha256:38dedb84b0e91806c106ca60b55be0
> c250b5dd7a9835fff8cf947dffc1891fa7
> > Deleted: sha256:fb08cf7282927f210c07fec3137a6a
> 924ca9907e74642d541f0d7befa3d1e4fd
> > Deleted: sha256:7e3d232c3781d955983847485fa908
> 73007156c796b77e68acd5ce76c2c20d32
> > Deleted: sha256:d1a76ebe400432a9c450bfcb736d6f
> b80f3f0438a66fabe5f1da8974de77978b
> > Deleted: sha256:de5a6ce4bd8621028ad3d9c720bc45
> 72a688d8e374445460acf8e6ab77b3ef78
> > Deleted: sha256:d9280a65cd3a7ec1484a8205d8025f
> eb23de44b081206a46822f5a212a7c6208
> > Deleted: sha256:afd34f01a7c302fbcb6e94a11f4da6
> 0a0addfc86d30ef3943a1a0787220a4fb6
> > Deleted: sha256:149bf2624cb44e56696305964dc352
> 4af173347601181391442d15b073fa0ae5
> > Deleted: sha256:cdab96d7c285fe42c3fe8d314e5c02
> 196d2bffbecdd9c26a5b2faba849e79d7c
> > Deleted: sha256:d0ef225f6403dbc4b68257bc21ae04
> f0b630aba21f05d0979a08f113b4a716d2
> > Deleted: sha256:d58e420fd0a88f80c893435a8ffc9e
> d7be68d1c87a50d53c460a625d8b83735b
> > Deleted: sha256:1c087b636db291826640ffb0d3ba26
> 672b43494151ba4eef5955b2121a655a31
> > Deleted: sha256:a07d988b44ae12da63d88902671fc5
> 953854715358c10d7b4eb34bf0438b51f5
> > Build step 'Execute shell' marked build as failure
>
>