You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Benjamin Mahler <bm...@apache.org> on 2018/05/30 01:11:14 UTC

Review Request 67374: Updated flags to support choosing the random sorter.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67374/
-----------------------------------------------------------

Review request for mesos, Gaston Kleiman, Greg Mann, and Meng Zhu.


Bugs: MESOS-8936
    https://issues.apache.org/jira/browse/MESOS-8936


Repository: mesos


Description
-------

This deprecates `--user_sorter` in favor of `--role_sorter`.

This patch also modifies the `--allocator` default to exclude "DRF"
from the value, while still ensuring the previous default works.

For now, both sorter flags must be set to `drf` or `random`.


Diffs
-----

  include/mesos/allocator/allocator.hpp e5a5355646c834280008867e89eb258eaefc2f1d 
  src/master/allocator/allocator.cpp e3dd7376b5f05e72d9b9aa895ed3aebc461e4372 
  src/master/constants.hpp b31e5f027b44d371050cea9a4662a4b54127812f 
  src/master/flags.hpp 4715f2bf4eb62808265f3062f93e370faa53e63d 
  src/master/flags.cpp 4ad13752719daf2cfad1f578edffc1f16a101760 
  src/master/main.cpp 3eb1f89cb877fdd84434954ed80e753b1f859338 


Diff: https://reviews.apache.org/r/67374/diff/1/


Testing
-------

make check


Thanks,

Benjamin Mahler


Re: Review Request 67374: Updated flags to support choosing the random sorter.

Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67374/#review204029
-----------------------------------------------------------



FAIL: Some of the unit tests failed. Please check the relevant logs.

Reviews applied: `['67369', '67370', '67371', '67372', '67373', '67374']`

Failed command: `Start-MesosCITesting`

All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67374

Relevant logs:

- [mesos-tests-cmake-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/67374/logs/mesos-tests-cmake-stdout.log):

```
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(124): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(128): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(279): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(301): warning C4267: 'initializing': conversion from 'size_t' to 'int', possible loss of data [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(368): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(372): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(512): warning C4267: 'function': conversion from 'size_t' to 'int', possible loss of data [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(543): warning C4996: 'strcpy': This function or variable may be unsafe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(548): warning C4996: 'fopen': This function or variable may be unsafe. Consider using fopen_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]
         d:\dcos\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8\src\c\src\cli.c(569): warning C4996: 'strcpy': This function or variable may be unsafe. Consider using strcpy_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8\src\zookeeper-3.4.8-build\cli.vcxproj] [D:\DCOS\mesos\3rdparty\zookeeper-3.4.8.vcxproj]


       "D:\DCOS\mesos\src\tests\mesos-tests.vcxproj" (default target) (1) ->
       (Link target) -> 
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: __cdecl mesos::internal::master::allocator::RandomSorter::RandomSorter(void)" (??0RandomSorter@allocator@master@internal@mesos@@QEAA@XZ) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual __cdecl mesos::internal::master::allocator::RandomSorter::~RandomSorter(void)" (??1RandomSorter@allocator@master@internal@mesos@@UEAA@XZ) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual void __cdecl mesos::internal::master::allocator::RandomSorter::add(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?add@RandomSorter@allocator@master@internal@mesos@@UEAAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual void __cdecl mesos::internal::master::allocator::RandomSorter::remove(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &)" (?remove@RandomSorter@allocator@master@internal@mesos@@UEAAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@Z) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual void __cdecl mesos::internal::master::allocator::RandomSorter::allocated(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &,class mesos::SlaveID const &,class mesos::Resources const &)" (?allocated@RandomSorter@allocator@master@internal@mesos@@UEAAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBVSlaveID@5@AEBVResources@5@@Z) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual void __cdecl mesos::internal::master::allocator::RandomSorter::unallocated(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const &,class mesos::SlaveID const &,class mesos::Resources const &)" (?unallocated@RandomSorter@allocator@master@internal@mesos@@UEAAXAEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@AEBVSlaveID@5@AEBVResources@5@@Z) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual void __cdecl mesos::internal::master::allocator::RandomSorter::add(class mesos::SlaveID const &,class mesos::Resources const &)" (?add@RandomSorter@allocator@master@internal@mesos@@UEAAXAEBVSlaveID@5@AEBVResources@5@@Z) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual void __cdecl mesos::internal::master::allocator::RandomSorter::remove(class mesos::SlaveID const &,class mesos::Resources const &)" (?remove@RandomSorter@allocator@master@internal@mesos@@UEAAXAEBVSlaveID@5@AEBVResources@5@@Z) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         sorter_tests.obj : error LNK2019: unresolved external symbol "public: virtual class std::vector<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::allocator<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > > __cdecl mesos::internal::master::allocator::RandomSorter::sort(void)" (?sort@RandomSorter@allocator@master@internal@mesos@@UEAA?AV?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@XZ) referenced in function "private: virtual void __cdecl mesos::internal::tests::CommonSorterTest_BENCHMARK_FullSort_Test<class mesos::internal::master::allocator::RandomSorter>::TestBody(void)" (?TestBody@?$CommonSorterTest_BENCHMARK_FullSort_Test@VRandomSorter@allocator@master@internal@mesos@@@tests@internal@mesos@@EEAAXXZ) [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]
         D:\DCOS\mesos\src\mesos-tests.exe : fatal error LNK1120: 9 unresolved externals [D:\DCOS\mesos\src\tests\mesos-tests.vcxproj]

    172 Warning(s)
    10 Error(s)

Time Elapsed 00:22:56.99
```

- Mesos Reviewbot Windows


On May 30, 2018, 1:11 a.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67374/
> -----------------------------------------------------------
> 
> (Updated May 30, 2018, 1:11 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Meng Zhu.
> 
> 
> Bugs: MESOS-8936
>     https://issues.apache.org/jira/browse/MESOS-8936
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This deprecates `--user_sorter` in favor of `--role_sorter`.
> 
> This patch also modifies the `--allocator` default to exclude "DRF"
> from the value, while still ensuring the previous default works.
> 
> For now, both sorter flags must be set to `drf` or `random`.
> 
> 
> Diffs
> -----
> 
>   include/mesos/allocator/allocator.hpp e5a5355646c834280008867e89eb258eaefc2f1d 
>   src/master/allocator/allocator.cpp e3dd7376b5f05e72d9b9aa895ed3aebc461e4372 
>   src/master/constants.hpp b31e5f027b44d371050cea9a4662a4b54127812f 
>   src/master/flags.hpp 4715f2bf4eb62808265f3062f93e370faa53e63d 
>   src/master/flags.cpp 4ad13752719daf2cfad1f578edffc1f16a101760 
>   src/master/main.cpp 3eb1f89cb877fdd84434954ed80e753b1f859338 
> 
> 
> Diff: https://reviews.apache.org/r/67374/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>


Re: Review Request 67374: Updated flags to support choosing the random sorter.

Posted by Benjamin Mahler <bm...@apache.org>.

> On June 1, 2018, 8:53 p.m., Meng Zhu wrote:
> > include/mesos/allocator/allocator.hpp
> > Line 65 (original), 68 (patched)
> > <https://reviews.apache.org/r/67374/diff/1/?file=2031739#file2031739line68>
> >
> >     I think exporting quotaRole sorter could also be useful, add a TODO?

Yeah, although I think if we were to do this we probably would need the quota guarantee sorter to be configurable at framework level?


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67374/#review204201
-----------------------------------------------------------


On May 30, 2018, 1:11 a.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67374/
> -----------------------------------------------------------
> 
> (Updated May 30, 2018, 1:11 a.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Meng Zhu.
> 
> 
> Bugs: MESOS-8936
>     https://issues.apache.org/jira/browse/MESOS-8936
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This deprecates `--user_sorter` in favor of `--role_sorter`.
> 
> This patch also modifies the `--allocator` default to exclude "DRF"
> from the value, while still ensuring the previous default works.
> 
> For now, both sorter flags must be set to `drf` or `random`.
> 
> 
> Diffs
> -----
> 
>   include/mesos/allocator/allocator.hpp e5a5355646c834280008867e89eb258eaefc2f1d 
>   src/master/allocator/allocator.cpp e3dd7376b5f05e72d9b9aa895ed3aebc461e4372 
>   src/master/constants.hpp b31e5f027b44d371050cea9a4662a4b54127812f 
>   src/master/flags.hpp 4715f2bf4eb62808265f3062f93e370faa53e63d 
>   src/master/flags.cpp 4ad13752719daf2cfad1f578edffc1f16a101760 
>   src/master/main.cpp 3eb1f89cb877fdd84434954ed80e753b1f859338 
> 
> 
> Diff: https://reviews.apache.org/r/67374/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>


Re: Review Request 67374: Updated flags to support choosing the random sorter.

Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67374/#review204201
-----------------------------------------------------------


Fix it, then Ship it!





include/mesos/allocator/allocator.hpp
Line 65 (original), 68 (patched)
<https://reviews.apache.org/r/67374/#comment286595>

    I think exporting quotaRole sorter could also be useful, add a TODO?



src/master/main.cpp
Line 337 (original), 339 (patched)
<https://reviews.apache.org/r/67374/#comment286596>

    I think it is still beneficial to print `flags.allocator` .


- Meng Zhu


On May 29, 2018, 6:11 p.m., Benjamin Mahler wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67374/
> -----------------------------------------------------------
> 
> (Updated May 29, 2018, 6:11 p.m.)
> 
> 
> Review request for mesos, Gaston Kleiman, Greg Mann, and Meng Zhu.
> 
> 
> Bugs: MESOS-8936
>     https://issues.apache.org/jira/browse/MESOS-8936
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> This deprecates `--user_sorter` in favor of `--role_sorter`.
> 
> This patch also modifies the `--allocator` default to exclude "DRF"
> from the value, while still ensuring the previous default works.
> 
> For now, both sorter flags must be set to `drf` or `random`.
> 
> 
> Diffs
> -----
> 
>   include/mesos/allocator/allocator.hpp e5a5355646c834280008867e89eb258eaefc2f1d 
>   src/master/allocator/allocator.cpp e3dd7376b5f05e72d9b9aa895ed3aebc461e4372 
>   src/master/constants.hpp b31e5f027b44d371050cea9a4662a4b54127812f 
>   src/master/flags.hpp 4715f2bf4eb62808265f3062f93e370faa53e63d 
>   src/master/flags.cpp 4ad13752719daf2cfad1f578edffc1f16a101760 
>   src/master/main.cpp 3eb1f89cb877fdd84434954ed80e753b1f859338 
> 
> 
> Diff: https://reviews.apache.org/r/67374/diff/1/
> 
> 
> Testing
> -------
> 
> make check
> 
> 
> Thanks,
> 
> Benjamin Mahler
> 
>