You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Benno Evers <be...@mesosphere.com> on 2017/12/11 20:26:30 UTC
Review Request 64507: Added a master flag to disallow agents without
domain.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/
-----------------------------------------------------------
Review request for mesos.
Bugs: MESOS-8115
https://issues.apache.org/jira/browse/MESOS-8115
Repository: mesos
Description
-------
Added a new `--require_agent_domains` flag and implementation. When
set to true, it will cause the master to refuse (re-)registration
attempts for agents with no configured domain.
This is intended as a safety net for operators, who could forget to
configure the fault domain of a remote agent and let it join the
cluster. If this happens, an agent in a remote region will be
considered a local agent by the master and frameworks (because agent's fault domain is not configured), causing tasks to potentially land in a remote agent which is undesirable.
Diffs
-----
docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
Diff: https://reviews.apache.org/r/64507/diff/1/
Testing
-------
`make check`
Thanks,
Benno Evers
Re: Review Request 64507: Added a master flag to disallow agents
without domain.
Posted by Vinod Kone <vi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/#review194087
-----------------------------------------------------------
Ship it!
Ship It!
- Vinod Kone
On Dec. 13, 2017, 2:49 p.m., Benno Evers wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64507/
> -----------------------------------------------------------
>
> (Updated Dec. 13, 2017, 2:49 p.m.)
>
>
> Review request for mesos.
>
>
> Bugs: MESOS-8115
> https://issues.apache.org/jira/browse/MESOS-8115
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Added a new `--require_agent_domain` flag and implementation. When
> set to true, it will cause the master to refuse (re-)registration
> attempts for agents with no configured domain.
>
> This is intended as a safety net for operators, who could forget to
> configure the fault domain of a remote agent and let it join the
> cluster. If this happens, an agent in a remote region will be
> considered a local agent by the master and frameworks (because agent's
> fault domain is not configured), causing tasks to potentially land in a
> remote agent which is undesirable.
>
>
> Diffs
> -----
>
> docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
> src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
> src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
> src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
> src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
>
>
> Diff: https://reviews.apache.org/r/64507/diff/2/
>
>
> Testing
> -------
>
> `make check`
>
>
> Thanks,
>
> Benno Evers
>
>
Re: Review Request 64507: Added a master flag to disallow agents
without domain.
Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/#review193686
-----------------------------------------------------------
FAIL: Some Mesos tests failed.
Reviews applied: `['64506', '64507']`
Failed command: `D:\DCOS\mesos\src\mesos-tests.exe --verbose`
All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64507
Relevant logs:
- [mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64507/logs/mesos-tests-stdout.log):
```
[----------] 1 test from IsolationFlag/CpuIsolatorTest
[ RUN ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0
[ OK ] IsolationFlag/CpuIsolatorTest.ROOT_UserCpuUsage/0 (2313 ms)
[----------] 1 test from IsolationFlag/CpuIsolatorTest (2335 ms total)
[----------] 1 test from IsolationFlag/MemoryIsolatorTest
[ RUN ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0
[ OK ] IsolationFlag/MemoryIsolatorTest.ROOT_MemUsage/0 (2238 ms)
[----------] 1 test from IsolationFlag/MemoryIsolatorTest (2262 ms total)
[----------] Global test environment tear-down
[==========] 829 tests from 84 test cases ran. (307339 ms total)
[ PASSED ] 819 tests.
[ FAILED ] 10 tests, listed below:
[ FAILED ] OfferOperationStatusUpdateManagerTest.UpdateAndAckNonTerminalUpdate
[ FAILED ] OfferOperationStatusUpdateManagerTest.RecoverCheckpointedStream
[ FAILED ] OfferOperationStatusUpdateManagerTest.RecoverEmptyFile
[ FAILED ] OfferOperationStatusUpdateManagerTest.RecoverTerminatedStream
[ FAILED ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdate
[ FAILED ] OfferOperationStatusUpdateManagerTest.IgnoreDuplicateUpdateAfterRecover
[ FAILED ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAck
[ FAILED ] OfferOperationStatusUpdateManagerTest.RejectDuplicateAckAfterRecover
[ FAILED ] OfferOperationStatusUpdateManagerTest.NonStrictRecoveryCorruptedFile
[ FAILED ] SlaveTest.ResourceProviderPublishAll
10 FAILED TESTS
YOU HAVE 204 DISABLED TESTS
```
- [mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/64507/logs/mesos-tests-stderr.log):
```
I1213 15:37:13.940055 7484 master.cpp:10209] Updating the state of task 4534f1c2-0a02-420d-b2de-3565c9f44084 of framework aa6373c3-5cb9-4793-bff6-622230ec8038-0000 (latest state: TASK_KILLED, status update state: TASK_KILLED)
I1213 15:37:13.940055 7088 slave.cpp:3400] Shutting down framework aa6373c3-5cb9-4793-bff6-622230ec8038-0000
I1213 15:37:13.941035 7088 slave.cpp:6114] Shutting down executor '4534f1c2-0a02-420d-b2de-3565c9f44084' of framework aa6373c3-5cb9-4793-bff6-622230ec8038-0000 at executor(1)@10.3.1.5:50552
I1213 15:37:13.941035 7088 slave.cpp:909] Agent terminating
W1213 15:37:13.942034 7088 slave.cpp:3396] Ignoring shI1213 15:37:13.286041 2436 exec.cpp:162] Version: 1.5.0
I1213 15:37:13.310042 8368 exec.cpp:237] Executor registered on agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0
I1213 15:37:13.313041 1952 executor.cpp:171] Received SUBSCRIBED event
I1213 15:37:13.317041 1952 executor.cpp:175] Subscribed executor on build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net
I1213 15:37:13.317041 1952 executor.cpp:171] Received LAUNCH event
I1213 15:37:13.321040 1952 executor.cpp:637] Starting task 4534f1c2-0a02-420d-b2de-3565c9f44084
I1213 15:37:13.398041 1952 executor.cpp:477] Running 'D:\DCOS\mesos\src\mesos-containerizer.exe launch <POSSIBLY-SENSITIVE-DATA>'
I1213 15:37:13.912034 1952 executor.cpp:650] Forked command at 6908
I1213 15:37:13.942034 7388 exec.cpp:435] Executor asked to shutdown
I1213 15:37:13.942034 8584 executor.cpp:171] Received SHUTDOWN event
I1213 15:37:13.942034 8584 executor.cpp:747] Shutting down
I1213 15:37:13.943034 8584 executor.cpp:854] Sending SIGTERM to process tree at pid 6utdown framework aa6373c3-5cb9-4793-bff6-622230ec8038-0000 because it is terminating
I1213 15:37:13.942034 7484 master.cpp:10315] Removing task 4534f1c2-0a02-420d-b2de-3565c9f44084 with resources cpus(allocated: *):4; mem(allocated: *):2048; disk(allocated: *):1024; ports(allocated: *):[31000-32000] of framework aa6373c3-5cb9-4793-bff6-622230ec8038-0000 on agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0 at slave(326)@10.3.1.5:50531 (build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 15:37:13.945034 7484 master.cpp:1305] Agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0 at slave(326)@10.3.1.5:50531 (build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1213 15:37:13.945034 7484 master.cpp:3364] Disconnecting agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0 at slave(326)@10.3.1.5:50531 (build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 15:37:13.945034 7484 master.cpp:3383] Deactivating agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0 at slave(326)@10.3.1.5:50531 (build-srv-04.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1213 15:37:13.945034 8196 hierarchical.cpp:344] Removed framework aa6373c3-5cb9-4793-bff6-622230ec8038-0000
I1213 15:37:13.945034 4920 containerizer.cpp:2337] Destroying container 33afc1c7-0a8c-42cf-b68e-550c419776c7 in RUNNING state
I1213 15:37:13.946033 4920 containerizer.cpp:2939] Transitioning the state of container 33afc1c7-0a8c-42cf-b68e-550c419776c7 from RUNNING to DESTROYING
I1213 15:37:13.946033 7820 hierarchical.cpp:766] Agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0 deactivated
I1213 15:37:13.946033 4920 launcher.cpp:156] Asked to destroy container 33afc1c7-0a8c-42cf-b68e-550c419776c7
I1213 15:37:13.954051 7484 containerizer.cpp:2788] Container 33afc1c7-0a8c-42cf-b68e-550c419776c7 has exited
I1213 15:37:13.982033 8412 master.cpp:1147] Master terminating
I1213 15:37:13.984055 7088 hierarchical.cpp:609] Removed agent aa6373c3-5cb9-4793-bff6-622230ec8038-S0
I1213 15:37:14.319053 3168 process.cpp:887] Failed to accept socket: future discarded
```
- Mesos Reviewbot Windows
On Dec. 13, 2017, 2:49 p.m., Benno Evers wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64507/
> -----------------------------------------------------------
>
> (Updated Dec. 13, 2017, 2:49 p.m.)
>
>
> Review request for mesos.
>
>
> Bugs: MESOS-8115
> https://issues.apache.org/jira/browse/MESOS-8115
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Added a new `--require_agent_domain` flag and implementation. When
> set to true, it will cause the master to refuse (re-)registration
> attempts for agents with no configured domain.
>
> This is intended as a safety net for operators, who could forget to
> configure the fault domain of a remote agent and let it join the
> cluster. If this happens, an agent in a remote region will be
> considered a local agent by the master and frameworks (because agent's
> fault domain is not configured), causing tasks to potentially land in a
> remote agent which is undesirable.
>
>
> Diffs
> -----
>
> docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
> src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
> src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
> src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
> src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
>
>
> Diff: https://reviews.apache.org/r/64507/diff/2/
>
>
> Testing
> -------
>
> `make check`
>
>
> Thanks,
>
> Benno Evers
>
>
Re: Review Request 64507: Added a master flag to disallow agents
without domain.
Posted by Benno Evers <be...@mesosphere.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/
-----------------------------------------------------------
(Updated Dec. 13, 2017, 2:49 p.m.)
Review request for mesos.
Changes
-------
Renamed flag to `--require_agent_domain`.
Bugs: MESOS-8115
https://issues.apache.org/jira/browse/MESOS-8115
Repository: mesos
Description (updated)
-------
Added a new `--require_agent_domain` flag and implementation. When
set to true, it will cause the master to refuse (re-)registration
attempts for agents with no configured domain.
This is intended as a safety net for operators, who could forget to
configure the fault domain of a remote agent and let it join the
cluster. If this happens, an agent in a remote region will be
considered a local agent by the master and frameworks (because agent's
fault domain is not configured), causing tasks to potentially land in a
remote agent which is undesirable.
Diffs (updated)
-----
docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
Diff: https://reviews.apache.org/r/64507/diff/2/
Changes: https://reviews.apache.org/r/64507/diff/1-2/
Testing
-------
`make check`
Thanks,
Benno Evers
Re: Review Request 64507: Added a master flag to disallow agents
without domain.
Posted by Benno Evers <be...@mesosphere.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/#review193438
-----------------------------------------------------------
docs/configuration/master.md
Lines 490 (patched)
<https://reviews.apache.org/r/64507/#comment271982>
Some notes about the etymology:
I considered `enforce_agent_domains`, but that sounds to me like the master should be actively doing something, i.e. at least send a ShutdownSlaveMessage.
I also considered `require_agent_domain`, but that sounds like the flag would restrict agents to one particular domain.
Apart from this, I don't really a strong opinion on how to call this.
- Benno Evers
On Dec. 11, 2017, 8:26 p.m., Benno Evers wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64507/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2017, 8:26 p.m.)
>
>
> Review request for mesos.
>
>
> Bugs: MESOS-8115
> https://issues.apache.org/jira/browse/MESOS-8115
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Added a new `--require_agent_domains` flag and implementation. When
> set to true, it will cause the master to refuse (re-)registration
> attempts for agents with no configured domain.
>
> This is intended as a safety net for operators, who could forget to
> configure the fault domain of a remote agent and let it join the
> cluster. If this happens, an agent in a remote region will be
> considered a local agent by the master and frameworks (because agent's fault domain is not configured), causing tasks to potentially land in a remote agent which is undesirable.
>
>
> Diffs
> -----
>
> docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
> src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
> src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
> src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
> src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
>
>
> Diff: https://reviews.apache.org/r/64507/diff/1/
>
>
> Testing
> -------
>
> `make check`
>
>
> Thanks,
>
> Benno Evers
>
>
Re: Review Request 64507: Added a master flag to disallow agents
without domain.
Posted by Vinod Kone <vi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/#review193451
-----------------------------------------------------------
`require_agent_domains` sounds weird because there is only one domain. i think `require_agent_domain` is fine.
src/tests/master_tests.cpp
Lines 8258 (patched)
<https://reviews.apache.org/r/64507/#comment271996>
move this right after `advance`.
src/tests/master_tests.cpp
Lines 8290 (patched)
<https://reviews.apache.org/r/64507/#comment271997>
s/an/and/
src/tests/master_tests.cpp
Lines 8291 (patched)
<https://reviews.apache.org/r/64507/#comment271998>
`ReregisterSlaveMessage`
src/tests/master_tests.cpp
Lines 8316 (patched)
<https://reviews.apache.org/r/64507/#comment271999>
move this after `post`
- Vinod Kone
On Dec. 11, 2017, 8:26 p.m., Benno Evers wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64507/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2017, 8:26 p.m.)
>
>
> Review request for mesos.
>
>
> Bugs: MESOS-8115
> https://issues.apache.org/jira/browse/MESOS-8115
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Added a new `--require_agent_domains` flag and implementation. When
> set to true, it will cause the master to refuse (re-)registration
> attempts for agents with no configured domain.
>
> This is intended as a safety net for operators, who could forget to
> configure the fault domain of a remote agent and let it join the
> cluster. If this happens, an agent in a remote region will be
> considered a local agent by the master and frameworks (because agent's fault domain is not configured), causing tasks to potentially land in a remote agent which is undesirable.
>
>
> Diffs
> -----
>
> docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
> src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
> src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
> src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
> src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
>
>
> Diff: https://reviews.apache.org/r/64507/diff/1/
>
>
> Testing
> -------
>
> `make check`
>
>
> Thanks,
>
> Benno Evers
>
>
Re: Review Request 64507: Added a master flag to disallow agents
without domain.
Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64507/#review193452
-----------------------------------------------------------
Bad review!
Error:
No reviewers specified. Please find a reviewer by asking on JIRA or the mailing list.
- Mesos Reviewbot Windows
On Dec. 11, 2017, 12:26 p.m., Benno Evers wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64507/
> -----------------------------------------------------------
>
> (Updated Dec. 11, 2017, 12:26 p.m.)
>
>
> Review request for mesos.
>
>
> Bugs: MESOS-8115
> https://issues.apache.org/jira/browse/MESOS-8115
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Added a new `--require_agent_domains` flag and implementation. When
> set to true, it will cause the master to refuse (re-)registration
> attempts for agents with no configured domain.
>
> This is intended as a safety net for operators, who could forget to
> configure the fault domain of a remote agent and let it join the
> cluster. If this happens, an agent in a remote region will be
> considered a local agent by the master and frameworks (because agent's fault domain is not configured), causing tasks to potentially land in a remote agent which is undesirable.
>
>
> Diffs
> -----
>
> docs/configuration/master.md 6af16a08257680963884d07a325803487f677c37
> src/master/flags.hpp edda71af41a8d199bb24b84487300e9c85b12fec
> src/master/flags.cpp 18f405b7e9cc6bd0011284b631974eee52dd2bf3
> src/master/master.cpp b3e074cfe86600793310deb87932fa145e95055d
> src/tests/master_tests.cpp 7b087675200fe0ca69319b715997700a91d9f6b2
>
>
> Diff: https://reviews.apache.org/r/64507/diff/1/
>
>
> Testing
> -------
>
> `make check`
>
>
> Thanks,
>
> Benno Evers
>
>